Computer implemented methods and systems for generating virtual body models for garment fit visualisation

ABSTRACT

Methods for generating and sharing a virtual body model of a person, created with a small number of measurements and a single photograph, combined with one or more images of garments. The virtual body model represents a realistic representation of the users body and is used for visualizing photo-realistic fit visualizations of garments, hairstyles, make-up, and/or other accessories. The virtual garments are created from layers based on photographs of real garment from multiple angles. Furthermore the virtual body model is used in multiple embodiments of manual and automatic garment, make-up, and, hairstyle recommendations, such as, from channels, friends, and fashion entities. The virtual body model is sharable for, as example, visualization and comments on looks. Furthermore it is also used for enabling users to buy garments that fit other users, suitable for gifts or similar. The implementation can also be used in peer-to-peer online sales where garments can be bought with the knowledge that the seller has a similar body shape and size as the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/000,088, filed Dec. 27, 2013, which claims the priority ofPCT/GB2012/050365, filed on Feb. 17, 2012, which claims priority toGreat Britain Application No. 1102794.3, filed on Feb. 17, 2011; GreatBritain Application No. 1111464.2, filed on Jul. 5, 2011; Great BritainApplication No. 1112931.9, filed on Jul. 27, 2011; and Great BritainApplication No. 1118467.8, filed on Oct. 25, 2011, the entire contentsof all of which are hereby incorporated in total by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The field of the invention relates to methods for generating and sharinga virtual body model of a person combined with an image of a garment,methods for generating an image of a user in a garment, methods forautomatically generating garment size recommendations, methods forvisualising generating make-up and hairstyle recommendations, methods ofgenerating a virtual body model of a user, methods for sharing a virtualbody model of a person, and methods of enabling users to interact withvirtual body models. The field of the invention includes systems whichrelate to these methods.

2. Technical Background

When shopping for clothes, users typically have entered shops whichspecialize in selling clothes, in order to try on the clothes beforepurchase. But the shops are usually closed for more than 12 hours perday, which limits shopping time. More recently, users have been able topurchase clothes from online retailers. During specification of theclothing for online purchase, users can specify a size, but they areunable to try the clothes on without first taking delivery of theclothes. If the clothes are not of the desired size or do not providethe hoped-for look, they may be returned, which entails expense. Currentestimates are that return rates for clothes bought online can be as highas 30% —largely because returned clothes do not fit. So at the presenttime, to try the clothes on, the user must either go to the shop, ormust wait for the clothes to be delivered, both of which take time andentail travel or delivery costs. It would be helpful if the user couldtry the clothes on in some way without having to travel to a shop, orhaving to wait to take delivery of the clothes. Considerable effortshave been made in recent years to provide computer-implemented systemsthat construct a virtual body model for a user—i.e. a virtual orcomputer-graphics based model of most (in some cases, all) of user'shead and body; such models are ideally meant to accurately portray theuser. These systems then provide a collection of virtual garments that,with varying degrees of accuracy, reflect the actual shape and size of aphysical garment that can be bought be a user. A garment can be selectedby the user and then fitted, or visualised, onto the user's virtual bodymodel. This enables the user to see what the garment would look like; inparticular, whether that style of garment suits the user and whether thefit for that specific size of garment is correct. However, such systemshave failed to provide a complete and practical solution. The variousaspects of this invention aim to address that failure.

3. Discussion of Related Art

WO2011033258A1 entitled SYSTEM AND METHOD FOR IMAGE PROCESSING ANDGENERATING A BODY MODEL, which is incorporated by reference, disclosesthe following. Images of foreground objects in a scene are generated bycausing electromagnetic radiation to be emitted having a first spectralpower distribution from a surface of a first foreground object, which isadjacent or at least partially obscured by a second foreground object. Afirst image of both of the first and second foreground objects isacquired whilst the first foreground object emits electromagneticradiation with the first spectral power distribution. A second image ofthe first and second foreground objects is acquired whilst the firstforeground object is not emitting electromagnetic radiation or isemitting electromagnetic radiation with a second spectral powerdistribution which is different to the first spectral powerdistribution. An alpha matte of the first and second foreground objectsis generated based on a comparison of the first image and second image.

WO2011033258A1 further discloses a method of generating a body model,comprising:

(i) defining at least one standard body model control point and/orstandard control measurement on at least one standard body model; and

(ii) generating a subject body model by defining at least one subjectcontrol point and/or subject control measurement corresponding to eachstandard body model control point and/or standard control measurement ina subject body model corresponding to a subject body, wherein step (ii)further comprises step (ii-1) of generating at least one subject mappingof each standard body model control point and/or standard controlmeasurement to its corresponding subject control point and/or subjectcontrol measurement.

WO2011033258A1 further discloses a method of generating a real life bodymodel image, comprising: defining at least one body model control pointon a body model image; defining a subject control point for each bodymodel control point in a subject image of a real life subject;generating a mapping each body model control point to its correspondingsubject control point; manipulating pixels of the body model image basedon the mapping so that pixels in the body model image align with pixelsin the subject image, thereby generating a manipulated real life bodymodel image.

Japanese patent application publication nos. 04037383, 11073491 andEuropean patent application publication no. 1909493 describeconventional systems whereby a background planar object and a foregroundobject (of any shape), which are located at different distances from animaging device, are discriminated from each other by illuminating thescene with different radiation frequencies. These documents describethat the foreground object must be positioned sufficiently far in frontof the background so that background and foreground lighting can betreated as independent, thereby allowing the foreground portions to bedistinguished from the background portions. Such systems do not permitforeground objects located near to and overlapping each other in theforeground part of the scene causing occlusion, to be readilydiscriminated from each other.

SUMMARY OF THE INVENTION

The invention provides computer implemented methods and systems forgenerating virtual body models for garment fit visualisation.

Further aspects and optional implementation features are defined in theclaims. Given the large number of aspects, we briefly summarise thembelow to aid in understanding the breadth and scope of these aspects.Note that each aspect is both independent of any other aspects, but canalso optionally be combined with any or all of the other aspects.

The computer-implemented system of the invention generates a user'svirtual body model, that is useable across multiple retail websites, andin which any of those websites enable or permit the user to combinevirtual garments, created from photographing physical garments, onto thevirtual body model.

The system enables a user to estimate how well a garment, available on apeer-to-peer web site, will fit that user, based on a comparison betweenthe seller's and the user's own body size measurements.

The system provides a ‘direct marketing engine’, automatically andwithout user intervention, pushing images to a customer that shows thatcustomer's virtual body model dressed in garments (chosen by sizerecommendation or otherwise)

The system models clothes that do not fit—the system has ability to showon the user's virtual body model how clothes that are a size too big ortoo small would look

The system includes a hairstyle recommendation engine, based on facialgeometry analysis and matching a hairstyle from a database of hairstylesto that facial geometry

The system automatically determines appropriate make-up descriptionsgiven the specific garment or garments selected by the user and thenprovides an image of the virtual body model combined with one or more ofthe garments, plus the make-up description

The system manipulates a virtual garment that is comprised of multiplelayers of 2D sprites, to generate 3d photo-real virtual clothing,morphed to match the virtual body model shape

The systems provides a UI for end-user's devices, in which there is animage of a user's virtual body model on a touch screen and with a (dragor flick) touch action different clothes are moved onto the virtual bodymodel.

The system can dynamically alter height and pose of the virtual bodymodel, depending on shoe or heel height.

The system uses a face photo to generate skin coloration and texture forthe body and that skin colouration is used as a filter or criteria whenthe system recommends a colour for a garment, or rates the colour of agarment already selected by the user—e.g. in terms of complimenting theuser's skin and/or eye colour.

The system supports 2 Screen TV—namely where a TV program featuresclothes, then the system automatically displays those same clothes on a‘second screen’ portable device (e.g. a tablet computer) visualised onthe virtual body model of the user. The user can view, interact with andbuy those garments from the portable device.

The system enables social network (e.g. Facebook®) interaction andintegration: the virtual body model is generated by a person interactingwith a web site, and the virtual body model is accessible or useable bya different web site, namely a social networking web site.

The system provides a web browser or browser extension; the systemautomatically identifies garments on browsed sites and can automaticallypresent the item on the user's virtual body model.

The system enables virtual body model sizing information to be used insearch engine's search algorithms

The system generates standardised garment sizing tables, scales orcharts, based on virtual body model data from large numbers of users'virtual body models.

The system provides a shared, live fitting room where different userscan see one or several virtual body models being dressed and dress none,one or several of the models depending on collaboration level.

The system needs only a small number of body measurements annotating asingle photograph to enable the system to create an accurate 3d bodymodel:top of the head, bottom of heels, crotch height, width of waist,width of hips, width of chest.

The system enables an icon, barcode on a garment label or advertisement,when scanned by a user's mobile computing device, to automatically causeimages of the garment to be combined onto the user's virtual body modelto enable that user to visualise what the garment will look like whenworn by them

The system uses depth stereo to improve post-processing, namely toautomatically differentiate pixels in the image that correspond to agarment from pixels in the image that do not correspond to the garmentand to then cut the image of the garment from the background and thenuse that image in generating a virtual 3D image of the garment.

The system enables a user select from one of several differentrecommendation channels; when a specific channel is selected, then thesystem automatically selects a garment which the system determines issuitable given parameters of the virtual body model and then combines animage of that garment onto the virtual body model to enable that personto visualise what the garment will look like when worn by them.

The system selects a garment for a user which the system determines issuitable, given the history or record of previous garments purchased orviewed by other users with similar virtual body models and then combinesan image of that garment onto the virtual body model of the user toenable that user to visualise what the garment will look like when wornby them.

The system enables photo-realistic visualisation of a garment on avirtual body model to be created: the garment is photographed indifferent lighting conditions to enable the system to synthesize newlighting conditions which match those applying to the virtual bodymodel.

The system enables the virtual body model to be accessible or useable byany type of display device (e.g. television, tablet computer, mobiletelephone) that is operable to use an extensible image framework; thatimages of a garment can on any such display device be seen as combinedonto the virtual body model, to enable a user to visualise what thegarment will look like when worn by them.

The systems enables a method of manufacturing a garment, in which a userhas a virtual body model of themselves, the method includes the steps of(a) the user selecting a garment from an on-screen library of virtualgarments; (b) a processing system automatically generating an image ofthe garment combined onto the virtual body model, the garment beingsized automatically to be a correct fit; (c) the processing systemgenerating data defining how a physical version of that garment would besized to provide that correct fit; (d) the system providing that data toa garment manufacturer to enable the manufacturer to make a garment thatfits the user.

The systems generates an accurate virtual body model for a user based onthe actual photograph and/or measurements supplied by the user anddisplaying that virtual body model on screen, together with an optionthat shows what the virtual body model would look like with a specificgarment if the user lost/gained set amounts of weight or undertookdefined levels of exercise. Thresholds set to ensure that extreme andunhealthy re-shaping was not possible.

The system includes a virtual wardrobe including clothes previouslybought on-line and hence already in a user's virtual wardrobe, as wellas garments not originally bought online but subsequently matched to avirtual equivalent, in order to allow the user to dress the virtual bodymodel on any of her garments (could be to dress or combine with garmentsto buy).

The system can determine if a shopper is buying garments for themselvesor for someone else: (a) the shopper providing a virtual body model ofthemselves to the system; (b) the shopper purchases garments at anon-line or retail store of a garment retailer; (c) the system matchesthe size of the purchased garments to the virtual body model and thendetermines if the purchased garments are for the shopper or not.

The system provides a method of generating a virtual body model, inwhich a user takes a photograph of their body which is then processed bya computer system to generate and display a virtual body, together withan approximate silhouette of the body, and the user is able tomanipulate the border or edge of the silhouette to make a silhouettethat more accurately matches the outline of the virtual body.

The system provides a method of generating a virtual body model, inwhich a user takes a photograph of their body which is then processed bya computer system to generate and display a virtual body model using, atleast, the body image and the user is presented with controls thatenable the shape and/or measurements of the virtual body model to bealtered to more accurately match the user's real or perceived shapeand/or measurements, in which the user is presented with an on-screenfield or control that enables the user to provide feedback about theaccuracy of the virtual body model.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a device for measuring a garment's elastic response.

FIG. 2 shows the virtual fitting room may be used in-store, online, oron a mobile device.

FIG. 3 shows an in-store concept flow of the Metail ecosystem.

FIG. 4 shows an example of steps to create a body model.

FIG. 5 shows a mobile phone may be used to scan a barcode of clothes orproducts in-store and the garment can be linked to the virtual fittingroom.

FIG. 6 shows the user may access product details through a mobiledevice.

FIG. 7 shows a user may use the mobile device to select the product andview the garment on their own personal body model in full 3D.

FIG. 8 shows an example of a user sharing an outfit with close friendson Facebook.

FIG. 9 shows a user may publish her own in-store style to attempt to bevoted the most ‘liked’ outfit of the month.

FIG. 10 shows an example of “My favourite look” in a social networkingapplication.

FIG. 11 shows examples of looks in a look book profile.

FIG. 12 shows how the consumer and retailer may be linked to core dataservices and channels.

FIG. 13 shows an overview of possible partner channels.

FIG. 14 shows possible connections between branded fitting rooms and aBig Co. which can provide a “virtual makeover”.

FIG. 15 shows a simplified example of a user journey.

FIG. 16 shows how a user may obtain different hairstyles on a virtualbody model.

FIG. 17 shows an example of eight views that a user can view a bodymodel from.

FIG. 18 shows eight photo positions which correspond to FIG. 17.

FIG. 19 shows alternative ways of entering measurements for bra size,with hips and waist size as options.

FIG. 20 shows alternative dimensions to measure and/or let the userenter body measurements.

FIGS. 21A and 21B illustrate a way in which two different layers (sleeveand torso), can be stretched or shrunk in order to match the body shapethey are “worn” on.

FIG. 22 shows a shoulder attachment position and a possible location ofan aperture.

FIG. 23 shows an example of a start page for a user, including an optionwhich is provided for the user to start the process of creating a bodymodel.

FIG. 24 shows that a user can zoom-in to the body model image and seethe body model and the garment in closer detail.

FIG. 25 shows an example of post processing after the upload of a photo,in which the picture is overlaid with lines which are to be adjusted bythe user according to instructions.

FIG. 26 shows an example of a body model which has been generated aftera user has completed the post processing and selected to continue.

FIG. 27 shows an example of a shared “My Look” in social media.

FIG. 28 shows an example of photography process and equipment set-up.

FIG. 29 shows an example of the use of a photography tool in processinglong sleeved garments.

FIG. 30 shows an example of a photography tool applied in processingtranslucent garments.

FIG. 31 shows examples of a photography tool applied in processingshoes.

FIG. 32 shows an example of a list for photography specification andrelated product information.

FIG. 33 shows an example of a step in a photography process.

FIG. 34 shows an example of metadata for hairstyles.

FIG. 35 shows an example of metadata for hairstyles.

FIG. 36 shows an example in which a set of different skin tones ispresented and the user is asked to select the skin tone that is closestto user skin tone in the photo.

FIG. 37 shows an example of a way of presenting different options forhairstyles, from which users may choose.

FIG. 38 shows an example of a matrix for garment layering.

FIG. 39 shows an example of a method for selecting differentbackgrounds.

FIG. 40 shows an example of a method for selecting different hairstyles.

FIGS. 41A and 41B show examples of stress-strain curves which may berelevant to garment stretch and to modelling garments on different bodymodels.

FIG. 42 show an example in which a garment section is larger than a bodysection.

FIGS. 43A and 43B show examples of garment stretch and modellinggarments on different body models.

FIG. 44 shows an example of IT infrastructure which may be used ingarment body modelling.

FIG. 45 shows examples of IT infrastructure which may be used in garmentbody modelling.

FIG. 46 shows an example of a method of selecting garments for dressinga body model.

DETAILED DESCRIPTION

The invention provided by Metail Ltd has multiple independent aspectsand optional implementation features and each will be described in turn.The virtual body model generated by the Metail system is called a ‘MeModel’.

As a preliminary overview, this Detailed Description includes thefollowing sections which describe key features of the Metail computerimplemented system that performs the various aspects and features ofthis invention:

-   Section 1: KEY FEATURES OF THE CONSUMER PRODUCT OR EXPERIENCE-   Section 28: KEY FEATURES OF THE DEVELOPMENT AND/OR DEMONSTRATION    FEATURES-   Section 57: KEY FEATURES USED IN BACK-END PROCESSING AND PHOTOGRAPHY-   Section 79: KEY FEATURES OF SOCIAL MEDIA INTEGRATION-   Section 109: KEY FEATURES OF THE VIRTUAL BODY MODEL-   Section 116: KEY FEATURES OF THE APPLICATION PROGRAMMING INTERFACE    AND MOBILE ASPECTS-   Section 121: KEY FEATURES OF DATA SERVICES-   Section 126: KEY FEATURES OF GARMENT MORPHING STRATEGIES-   Section 129: KEY FEATURES OF HEAD MODELLING-   Section 133: SERVER INFRASTRUCTURE-   Section 150: VARIOUS MISCELLANEOUS IMPLEMENTATION FEATURES

The complete list of section headings is as follows:

-   Section 1. KEY FEATURES OF THE CONSUMER PRODUCT OR EXPERIENCE-   Section 2. The overall ecosystem-   Section 3. User Experience Example-   Section 4. Creating a head from an uploaded photo-   Section 5. Selecting the hairstyle-   Section 6. Creating the virtual body model-   Section 7. Creating the virtual body model from an uploaded photo-   Section 8. Creating the body model without uploading a photo-   Section 9. The virtual fitting room-   Section 10. Different views of a garment-   Section 11. The garment overview in the virtual fitting room-   Section 12. The first time the user is presented with their virtual    body model-   Section 13. Show nudity level of the body model-   Section 14. Entering values on sliding scale updates the body model    live-   Section 15. Automatically adapting the virtual body model to    different heel heights-   Section 16. Automatically adjusting hem length and body model height-   Section 17. Changing hairstyle on the virtual body model-   Section 18. Using different backgrounds for the virtual body model-   Section 19. Lighting effects on the virtual body model-   Section 20. Automatic effects depending on garments-   Section 21. Photo effects applied to the garment on a virtual body    model-   Section 22. Connecting with other sites-   Section 23. The user's wardrobe-   Section 24. Level of body model completion-   Section 25. Randomized outfits-   Section 26. Informed randomized outfits-   Section 27. User satisfaction survey-   Section 28. KEY FEATURES OF THE DEVELOPMENT AND/OR DEMONSTRATION    FEATURES-   Section 29. Recommendation features-   Section 30. ‘Cover Flow’ type UX for garments-   Section 31. Automatic suggestion of outfit-   Section 32. Virtual fitting room-   Section 33. Cross promotion of goods-   Section 34. Body size matching and indexing for improved peer to    peer clothing purchasing-   Section 35. Using information about linked garments-   Section 36. In-store concepts-   Section 37. Personalized fashion tips based on your body size-   Section 38. Style Radio-   Section 39. Maternity clothing feature-   Section 40. Compare two body models with different outfits-   Section 41. Direct marketing including a picture of your body model    in selected clothes-   Section 42. Real avatar-   Section 43. Bespoke tailoring on body model-   Section 44. Aspirational modelling-   Section 45. Hairstyle recommendation engine-   Section 46. Try out different makeup-   Section 47. Beauty recommendations-   Section 48. Beauty recommendation engine-   Section 49. Facial hair-   Section 50. Glasses-   Section 51. Tattoo-   Section 52. Two screen TV-   Section 53. Styling room for publishers-   Section 54. Ageing body model-   Section 55. Give the body model a tan-   Section 56. Testing of new looks for future collections-   Section 57. KEY FEATURES USED IN BACK-END PROCESSING AND PHOTOGRAPHY-   Section 58. Back end processing-   Section 59. The 2.5D solution-   Section 60. Metail digitization process-   Section 61. Multi layer garments-   Section 62. Deforming mannequin-   Section 63. Digitization-   Section 64. Garment positioning-   Section 65. Change the model size and see the garment fit-   Section 66. Body model-   Section 67. How to turn weight and height or other input data into a    shape for the body model-   Section 68. Garment size and stretch-   Section 69. Garment sizing table-   Section 70. Providing retailers with specific size recommendation    tables-   Section 71. Adapting garment modelling to match with the user's body    model-   Section 72. Shadows-   Section 73. Photography Process and Tool Example-   Section 74. Mannequin adaptation-   Section 75. Garment segmentation and alternative methods-   Section 76. Garment translucency-   Section 77. How Metail uses metadata for hairstyles-   Section 78. How the virtual fitting room use metadata for hairstyles-   Section 79. KEY FEATURES OF SOCIAL MEDIA INTEGRATION-   Section 80. Social interaction-   Section 81. Social needs-   Section 82. Purely social-   Section 83. Exhibition-   Section 84. Recommendations-   Section 85. Discovery-   Section 86. Online social interaction-   Section 87. User profile-   Section 88. Social interaction features-   Section 89. Sharing the private wardrobe-   Section 90. Circle of friends-   Section 91. Get live feedback on look from friends-   Section 92. Fashion Panel AKA style counsel-   Section 93. Shared live fitting room-   Section 94. Instant chat through Facebook (FB) chat-   Section 95. Invite friends to shop with reminder-   Section 96. See which Facebook friends are using the virtual fitting    room-   Section 97. Become a stylist and be rated by peers-   Section 98. Stylist recommended according to style data-   Section 99. Try that look on me-   Section 100. Follow friends/celeb looks-   Section 101. Like and dislike stylist recommendations to find best    stylist for you-   Section 102. Look recommendations connected to a specific event-   Section 103. Different social platforms to share via-   Section 104. Email-   Section 105. Share alternative looks-   Section 106. Share the outfit-   Section 107. Sociability-   Section 108. Share your model-   Section 109. KEY FEATURES OF THE VIRTUAL BODY MODEL-   Section 110. Aiding body shape prediction through specific features    (pregnant etc.)-   Section 111. Improved body modelling using aggregated customer data-   Section 112. Modelling clothes that do not fit-   Section 113. Creating the face-   Section 114. Skin textures and use of face photo to generate skin    coloration for a body-   Section 115. Male vs. female model-   Section 116. KEY FEATURES OF THE APPLICATION PROGRAMMING INTERFACE    AND MOBILE ASPECTS-   Section 117. Mobile-   Section 118. Application Programming Interface (API)-   Section 119. Automatic detection of garments and method and system    to try the garments on the body model-   Section 120. The similarity system-   Section 121. KEY FEATURES OF DATA SERVICES-   Section 122. Uses for the data that is gathered-   Section 123. Market research-   Section 124. Size and shape demographic-   Section 125. Online Search-   Section 126. KEY FEATURES OF GARMENT MORPHING STRATEGIES-   Section 127. Photographing a deforming mannequin to capture shape    variations-   Section 128. Garment size and stretch-   Section 129. KEY FEATURES OF HEAD MODELLING-   Section 130. Attaching the face to the head model-   Section 131. Hair as a layered garment-   Section 132. Hands as a layered garment-   Section 133. SERVER INFRASTRUCTURE-   Section 134. Image/Me model rendering on the back server-   Section 135. Visualization Subsystem (VS)-   Section 136. Adding more nodes-   Section 137. Caching strategies-   Section 138. Two-part request affinity-   Section 139. Per-avatar session affinity-   Section 140. Client-side image caching-   Section 141. Garment data-   Section 142. Sharding-   Section 143. Distributed database (DB)-   Section 144. Hardware-   Section 145. Web tier-   Section 146. Improved DB performance-   Section 147. Optimize and Distribute performance critical code-   Section 148. Offload SSL decryption-   Section 149. Spare capacity/Node provisioning-   Section 150. VARIOUS MISCELLANEOUS IMPLEMENTATION FEATURES

Section 1: Key Features of the Consumer Product or Experience

There is provided a method of generating a virtual body model, in whicha user takes, or has taken for them, an image of their body which isthen processed by a computer system to generate and display a virtualbody, together with an approximate silhouette of the body, and the useris able to manipulate the border or edge of the silhouette to make asilhouette that more accurately matches the outline of the virtual body.

The method may be one in which the silhouette is displayed as beingover-laid over the image of the virtual body. The method may be one inwhich the silhouette is generated using image-processing techniquesapplied to a 2D photographic image of the user's body. The method may beone in which the user's body is imaged using a depth sensor thatincludes an infrared laser projector combined with a sensor whichcaptures video or still data in 3D.

There is also provided a method of generating a virtual body model, inwhich a user takes, or has taken for them, an image of their body whichis then processed by a computer system to generate and display a virtualbody model using, at least, the body image and the user is presentedwith controls that enable the shape and/or measurements of the virtualbody model to be altered to more accurately match the user's real orperceived shape and/or measurements, in which the user is presented withan on-screen field or control that enables the user to provide feedbackabout the accuracy of the virtual body model.

The method may be one in which the user is presented with on-screencontrols that allow the user to directly manipulate the shape of thevirtual body model.

There is also provided a method for enabling garments to be visualisedon a virtual body model of a user, in which (a) a display device showsan image of a garment, of a size selected by the user, combined onto thevirtual body model to enable the user to visualise what the garment willlook like when worn by them and (b) the display device also provides anicon, button, function, sliding scale or other control that, if selectedby the user, causes the virtual body model to be altered to show what itwould look like with that specific garment if the user lost/gained setamounts of weight defined on a user controllable scale.

The method may be one in which the display device also provides an icon,button, function, sliding scale or other control that, if selected bythe user, causes the virtual body model to be altered to show what itwould look like with that specific garment if the user altered theirbody shape through levels of exercise defined on a user controllablescale. The method may be one in which thresholds ensure that extreme andun-healthy re-shaping is not possible. The method may be one in whichthe control is voice activated using a speech recognition system.

There is also provided a method of generating a virtual body model of auser, including the legs and feet of the user, in which the user canselect a pair of shoes or boots for their virtual body model from aselection of footwear shown on-line, and a computer-implemented processretrieves the heel height for the selected footwear and automaticallyadjusts the height of the heel of the virtual body model depending onthe retrieved heel height.

The method may be one in which the virtual body model allows the virtualfoot to pivot about its virtual ankle and the heel height of the virtualbody model above a virtual floor is adjusted by pivoting the virtualfoot about the virtual ankle to give a desired height. The method may beone in which raising the heel of the virtual body model automaticallyalters the posture of the virtual body model. The method may be one inwhich the posture of the virtual body model is altered when the heel israised by tilting the virtual pelvis forward. The method may be one inwhich the user can choose for the virtual body model to be bare foot.The method may be one in which the bottom of a garment in relation to avirtual floor is raised as the virtual heel height is raised, but theposition of the bottom of the garment in relation to the user's legs isnot altered.

There is provided a method of generating photo-realistic images of agarment combined onto a virtual body model, in which a physical garmentis photographed in different lighting conditions and digital images forthe garment in each of those different lighting conditions is stored ona database; and a computer-based image processing system generates animage of the garment combined onto the virtual body model by selectingfrom the database, or permitting the selection from the database, ofgarment images lit in lighting conditions similar to the lightingconditions applying to the virtual body model and/or a background forthe virtual body model.

The method may be one in which the virtual body model includes an imageof a user's face, obtained from a digital photograph provided by theuser. The method may be one in which the lighting conditions include oneor more of: main direction of the lighting; colour balance of thelighting; colour temperature of the lighting; diffusiveness of thelighting. The method may be one in which the type of garment determinessimulated weather conditions, such as rain, sunshine, snow, which arethen applied to the image of the garment when combined with the virtualbody model. The method may be one in which a user can manually selectimages from the database by operating a control that mimics the effectof changing the lighting conditions in which a garment was photographed.The method may be one in which the image processing system canautomatically detect parameters of the lighting conditions applying tothe digital face photograph supplied by the user can select matchinggarment images from the database.

There is also provided a method for generating and sharing a virtualbody model of a person combined with an image of a garment, in which thevirtual body model is generated by analysing and processing one or morephotographs of a user, and a garment image is generated by analysing andprocessing one or more photographs of the garment; and in which thevirtual body model is accessible or use-able by multiple differentapplications or multiple different web sites, such that images of thegarment can, using any of these different applications or web sites, beseen as combined onto the virtual body model to enable visualization ofwhat the garment will look like when worn.

There is also provided a method for generating and sharing a virtualbody model of a person combined with an image of a garment, in which thevirtual body model is generated by analysing and processing one or morephotographs of a user, and a garment image is generated by analysing andprocessing one or more photographs of the garment and that garment imageis shown in a virtual fitting room, and in which the virtual fittingroom is accessible or useable by multiple different applications ormultiple different web sites, such that images of the garment can, usingany of these different applications or web sites, be seen as combinedonto the virtual body model to enable visualization of what the garmentwill look like when worn.

The method may be one in which one or more of the different applicationsor web sites each displays, in association with the image of thegarment, a single icon or button which, when selected, automaticallycauses one or more images of the garment to be combined onto the virtualbody model to enable a person to visualize what the garment will looklike when worn by them. The method may be one in which the combinedimage is a 3D photo-real image which the user can rotate and/or zoom.The method may be one in which one of the different web sites is agarment retail web site and the garment is available for purchase fromthat web site. The method may be one in which one of the different websites is a fashion related web site associated with a print publication.The method may be one in which the image of the garment is generated byphotographing in 2D an actual garment from a number of viewpoints, on amannequin of known size and shape. The method may be one in which thegarment is photographed from between 5 and 12 different viewpointsaround the garment and the resulting photographs are analysed andprocessed to generate a 3D photo-real image of the garment that can beviewed from 360 degrees. The method may be one in which the image of agarment is a 3D photo-real image of the garment. The method may be onein which, for a given garment, only a single size of that garment isphotographed and the appearance of other sizes is calculated byextrapolating from that single size. The method may be one in which theprocess of extrapolating is based on measuring other sizes of thatgarment, or different garments, from the same manufacturer of thatgarment. The method may be one in which it is possible to show/share avirtual body model to another user so that other user can select and buyclothes to fit the user, whilst concealing actual measurements from thatother user.

There is provided a method of visualising garments on a virtual bodymodel of a user, comprising the steps of (a) storing digital images ofmultiple garments available for purchase; (b) storing a virtual bodymodel of a user that enables the user to visualise what those garmentswould look like on their virtual body model; (c) providing a virtualwardrobe or garment storage environment which stores images of allgarments purchased by the user after they have been shown when combinedwith the user's virtual body model; (d) including in the virtualwardrobe or garment storage environment images of other garments,purchased or selected by the user without having been combined with theuser's virtual body model.

The method may be one in which the other garments, purchased or selectedby the user without having been combined with the user's virtual bodymodel, are manually selected by the user viewing images of garments in adigital catalogue or collection of garment images. The method may be onein which the virtual wardrobe or garment storage environment stores alsoimages of outfits, comprising several garments. The method may be one inwhich the other garments, purchased or selected by the user withouthaving been combined with the user's virtual body model, can bepurchased by the user selecting an icon or function that is displayedtogether with the other garments.

In this document, a “body model” is a visualisation of a user's bodyshape, an “outfit” is a set of clothes (an “outfit” can for exampleconsist of one or several garments and accessories), a “look” is thebody model dressed in an outfit (a “look” can for instance be sharedincluding the background and effects applied to that look), and the“virtual fitting room” system as described in this document is in oneexample implemented by the company Metail Ltd. Reference to the genderof a user is to be understood to apply to both genders of user, exceptwhere the context implies otherwise.

Section 2: The Overall Ecosystem

The Metail online fitting room can be described as providing the userwith a body model created to represent her body shape and allowing theuser to dress the body model in different garments to model the size andfit of the garments. The user can also receive recommendations offashion and on size of specific garments.

The user's body model can be used across many platforms and beinteracted with through partner sites where the user can dress the bodymodel in for instance a specific retailer's garments.

As an illustration FIG. 12 shows an example of applications of thetechnology. FIG. 15 shows examples of partner channels. FIG. 14 showsexamples of interaction points with other brands.

The virtual fitting room is enabling the overall process of obtaining amodel of the user's body as a means of visualizing fit and thisinformation can be used for instance in facilitating online marketing.

Section 3: User Experience Example

FIG. 23 shows an example of a start page for the user. There is anoption for the user to start the process of creating a body model.

In an alternative example, a start page is provided in which the userhas the option to log on to the virtual fitting room or register as anew user. The user can select to login using eg. their Facebook accountand the virtual fitting room account will then be verified via Facebook.The platform supports connections with other platforms as well.

In an alternative example the user is prompted to log in to the virtualfitting room using a third party login such as the Facebook login. Thisstep allows the user to log in and pair the virtual fitting room accountwith for instance a Facebook account irrespective of if the userpreviously had logged in to the virtual fitting room. Should the useralready have a virtual fitting room account associated with thatFacebook account the user is notified about that. The user can select tocreate a new body model or use the already stored one. In one examplethe user's previous model is automatically presented to the user atlogin.

If the user already has created a body model and at a later stage wouldlike to connect that body model to a Facebook account, but the Facebookaccount already has a body model associated with it, the user will benotified about that. The user is prompted to select which of the bodymodels should be associated with the Facebook account.

As a new user the idea is that you just get started. You can thenregister and save the body model for later retrieval. The system alsoallows the user to create a body model without saving it. The user cancreate the body model and also try garments on the model withoutregistering. To retrieve the model for the next time they use theservice, the user needs to register. The model is then saved andassociated with that particular user account.

The user can further select to start the process of creating the bodymodel through clicking on “Get Started” (eg. in FIG. 23).

In one example the user is prompted with a message when they navigateaway from the virtual fitting room to another website asking if theywould like to register before leaving to be able to retrieve their bodymodel at a later time. The user then is given the option to register orlog in to associate their current body model with a user account.

When the user selects to get started with the process of creating a bodymodel the user can select two different options on how muchpersonalisation she would like of the body model. One option is toupload a photo of the user's face and the other option is to use asample photograph. The step of using a sample photograph can either givethe user the option to choose from a set of stored sample photographs orautomatically select a photograph for the user.

One alternative is to provide the user with the option to upload apicture of their head or to skip this step. If the user selects to skipthis step a head is chosen from a set of sample heads. On this page theuser is also presented with guidance about the photo they can upload togive the best experience.

The photo will be used to create the face features on the head of thebody model. For example the face will also provide an indication of theskin tone of the user.

In an example the user is presented with a set of alternative facephotos to chose from in a selection page. The user can then select oneof several sample photographs and the user can also see a standard bodymodel, which they can rotate in to different views.

Section 4: Creating a Head from an Uploaded Photo

If the user selects to upload a picture of their face, the systeminitiates a process for upload of a file to the server. The user ispresented with a popup where she selects the file to upload.

The user is presented with an option to rotate the picture if it is inthe wrong orientation. In one example the picture is enlarged tofacilitate the post processing so the user is more likely to selectpoints accurately.

The user is then presented with a page where she is asked to annotatethe photograph with specific points. This is to aid the process ofcombining the head and the face in the image. The annotation markers canin different examples have different colours, shapes and be of differentsizes.

An example of the steps of post processing is now discussed. In oneexample the process consists of the user making annotations for thenose, the chin, the ear-lobes, the eyes, the mouth and the jaw. Thesystem is in one example using the placement of the first point as anestimate for where the rest of the annotation points should be presentedto the user. The system may use machine learning to make more accuratesuggested placements of the annotation markers. This makes the postprocessing easier for the user.

The system can in one example derive from the whole or a subset of theuser base's previous placements of the different points where the firstannotation mark (here exemplified with the nose) is most likely to be.

The user can click and drag a marker, for example a cross, to thecorrect position representing in the first step the nose. The user ispresented with instructions and an exemplary image to make the processeasier.

When the user has selected the point for the first annotation, in thiscase the nose, she can select to go to the next annotation input screen.The user is then instructed to place the cross on the chin. The nextstep of the post processing is to mark the points for the earlobes. Theuser then drags and drops the two crosses to positions representing theearlobes. The next step in the post processing is to mark the eyes. Theuser is presented with four crosses, which are to be placed on each sideof each eye. The next step of the post processing is to mark the endpoints of the mouth. The user is presented with two crosses, which areto be placed on each side of the mouth. The last annotation step is tomark out the jaw. The user places the two crosses on the jawline at twopoints approximately halfway between the chin and the mouth. It is to beunderstood that in a different example the annotation steps can bepresented to the user in a different order.

The user is then, as shown for example in FIG. 36, presented with a setof different skin tones and is asked to select the skin tone that shethinks is closest to her skin tone in the photo.

The skin tone selection will filter what hairstyles the user will beable to chose from in the next step. The process of creating the headfrom the photo and a hairstyle is described elsewhere in this document.

In an alternative example the skin tone is automatically detected fromone or several points in the uploaded photo. The relevant hairstyles,i.e. hairstyles that fit the user's skin colour, are provided forselection by the user. Points are identified and used to match the skincolour with the hairline.

See also Section 112 and 128.

Section 5: Selecting the Hairstyle

The next step in creating the body model's head is to choose a hairstylefor the model. The user is presented with different options forapplicable hairstyles. A possible way of presenting the hairstyles isshown in FIG. 37. The user can filter the different hairstyles based ondifferent options. In the example shown in FIG. 37, the keywords arepresented to the left.

The hairstyles shown are presented in one view and an alternative viewis presented when hovering over the hairstyle image. The alternativeview could be from a different angle or for instance the hairstyle shownfrom a 180 degrees changed view (ie from the opposite direction).

FIG. 18 shows a number system for classifying view direction. In oneexample the standard view is view 3 and the alternative view is view 7.The different views are also shown in FIG. 17.

When the user has selected the hairstyle to be used, the head is createdin the background on the server and will be viewable by the user when itis ready. The user is prompted that the head is being created and thatshe will be notified when it is ready.

Section 6: Creating the Virtual Body Model

The next step is that the user is given options (eg. two) on how tocreate the body model. One option is for the user to enter measurementsof her body and then the body model will be created from thosemeasurements. An alternative option is that the user uploads a photo andannotates that photo and also provides the height and weight. The systemwill then create a body model representing the user.

See also Section 65.

Section 7: Creating the Virtual Body Model from an Uploaded Photo

The user is asked to upload a photo of herself in a doorway. The doorwayis used to set the perspective of the photo. The user is then presentedwith four corners connected with lines overlaid on the picture ofherself in the doorway. The user is requested to mark out the corners ofthe doorway she is standing against with the four points. This will givethe system input to adjust for the perspective of the photo. The photois then adjusted by the system to account for the perspective view.

The user is then taken through the post processing where the picture isoverlaid with lines which are to be adjusted according to instructions.The annotations provide input in to the process of creating the bodymodel.

The first adjustment is to mark out the height of the user in thepicture. This is done by the user moving one of two horizontal lines tomeet the top of the head of the person in the picture and a second lineto meet the heels of the person in the picture. The next annotation isto mark the top of the inside of the leg. The user adjusts a horizontalline vertically to the correct position.

The user is then taken through three horizontal measurements where sheis to adjust the end points of a horizontal line to the contour of thebody in the photograph. The points of measurement are fixed in thevertical plane and the user can only adjust the end points on thehorizontal plane. Three measured points represent the measurements forthe waist, the hips, and the chest.

The placement of the horizontal measurement lines is calculated as apercentage of the torso length. The input to this are the height endpoints provided by the user in a previous step and the point of the topof the inner leg. The lines are then placed at certain percentage pointsof the torso length. The three measured points give input for the bodymodel creation in that they provide an indication about the user's bodyshape.

The next step is where the user enters her height and weight, forexample as shown in FIG. 25. These are measurements the user generallyknows and can enter without needing to find a scales or a measuringtape. The user can select the unit and slide the handle on the bar tothe correct position. In another example the user enters the value innumbers into an input field.

In one example if the user is a female and the model to be created isfemale she then also enters the bra size. The user can then enter in theband size and the cup size. Since these measurements can vary dependingon the country, in one example the option to indicate in which country'ssize the values have been entered is provided. The user then selects tocontinue and the body model is generated. The user is presented with thebody model for example as shown in FIG. 26.

The body model is generated totally synthetically from measurementsentered and from the model as it was generated in the system. Moredetails are described elsewhere in this document.

Upon completion of the body model the user gets a notification that thephoto annotations may be inaccurate and that there is a possibility toenter measurements manually. If the user selects to change the size ofthe body model measurements the body model creation is prioritised onthe latest input; the photo or the measurements. If the photo is usedfor the measurements and the user the changes the measurements the modelwill be more accurate.

See also Section 66.

Section 8: Creating the Body Model without Uploading a Photo

If the user selects to create a body model without uploading a photo theuser is in one example presented with different measurements to beentered.

The user is presented with options to enter measurements in two steps.First the basic measurements height and weight can be entered on asliding scale with the option to change the units.

In one example if the user is a female and the model to be created isfemale she then also enters the bra size. The user can then enter in theband size and the cup size. Since these measurements can vary dependingon the country, in one example the option to indicate in which country'ssize the values have been entered is provided.

The second step provides for refined measurements. The user is presentedwith the option to enter additional measurements for instance taken witha tape measure. These measurements can be for hips, waist and chest butalso other measurements can be entered.

See also Section 66.

Section 9: The Virtual Fitting Room

In the fitting room the body model can be dressed in different garmentsand rotated to be shown from different angles. For example eight views,which the user can view the body model from, are shown in FIG. 17; adrawing of the photo positions is shown in FIG. 18. The user can in oneexample as a short cut click on “back view” to show the backside view ofthe body model when showing a look. The user can zoom the body modelimage and see the body model and the garment in closer detail as shownfor example in FIG. 24. The user can change the head and themeasurements of the body model from the store interface, for instance ifthe user would like to change to a different hairstyle.

The Garments

The garments in the virtual fitting room are categorized in differentcategories such as shoes, dresses, jacket etc. A garment can bepresented under several categories. The user can either click on thegarment to dress the body model in that garment or the user can drag anddrop the garment on the body model.

Buying Garments

The user can buy the garments via the partner retailer's site, and ifselecting to buy, the garment is placed in the shopping basket. In oneexample the user can buy garments in the interface of the virtualfitting room and complete the transaction without leaving thatexperience.

Garment Layering in Virtual Fitting Room

In the virtual fitting room the user can layer clothes together having ashirt under a jacket and a skirt under a long coat. However, not allclothes should be able to be worn on top of each other; for instance onesetting can be that you should not be allowed to have a tight top on topof a coat.

The garments are divided in different groups, some of which can be worntogether and some of which cannot be worn together. FIG. 38 shows anexample of a layering matrix of different garments which shows how theycan be worn together.

The ones that cannot be worn together can be substitutes: for instanceyou cannot wear a pair of trousers and a pair of shorts at the sametime. Selecting to dress the body model in a pair of trousers while thebody model is already dressed in a pair of shorts will result in theshorts being removed from the outfit and the trousers will replace them.

In one example the user has the option to decide whether a garment thatin the standard mode would replace the garment the body model is alreadywearing to instead be worn on top of it. It could for instance be a vesttop that either could be replaced by a t-shirt or it could instead beworn under the t-shirt.

The system allows for the garments to be worn together or on top of eachother. In one example the different garments the body model is wearingare shown in a list where the user can move a specific garment to moveit up or down in the layering hierarchy (i.e. the order which they areput on the body model). There can also be buttons which the user canpress to bring garments forwards or send backwards in the layeringhierarchy.

In one example the user can also drag the actual garments to be placedin a different layering order. The list of clothes also allows forindividual deletion of specific garments in one outfit.

The software can in an alternative example derive from where the garmentgraphically is located in relation to the body model. If for instance ashoulder of a dress is 10 cm outside of the body model then the dresswould probably not fit under a jacket and the garment should not belayered together with the dress.

Section 10: Different Views of a Garment

Different garments also have different features such as a shirt can betucked-in or un-tucked, or a jacket can be open or closed. The systemprovides for the user to have the choice to select different variantsfor different clothes the body model is wearing. A similar problem isthat a user might want to fold up the legs of a pair of jeans or that adress could be taken up to a desired length. The virtual fitting roomprovides for different options to facilitate this.

In one example there are different variants on how to wear a garment, ordifferent variants of the garment itself. The user would then select toshow a pair of jeans with the legs folded up for instance.

Section 11: The Garment Overview in the Virtual Fitting Room

Each garment will have information associated with it. The informationis about the brand, the retailer and the price among other things. Agarment information page may be shown. The garment information page alsoshows the recommended actual size, based on the body model'smeasurements. If the user selects “buy from retailer”, that recommendedsize is the size that will be forwarded to the retailers system. Thesizing information also provides user specific information in relationto the garment's size and the body model's size. The informationprovided could for instance be that the garment in size 8 will be tightover the hips and that size 10 will fit you.

Section 12: The First Time the User is Presented with their Virtual BodyModel

The first time the user is presented with the model is after they haveentered in their measurements. The user will then be presented with themodel as representing themselves. One alternative way of showing theuser the model is with more or less clothes the first time.

The body model can be shown in different clothes the first time the usersees it. An illustrative picture may be one in which the body modelwears underwear. The body model can instead for instance be shown in atight or loosely fitting garment, which covers more of the skin. Thisgarment is then removed when the user selects a different garment fromthe garment library and the body model is shown in that garment.

When the user then removes the selected garment the body model couldeither be shown in underwear or in the other garment, which the user waspresented on the body model in the first step.

It is possible for the user to select to show the body model inunderwear.

Section 13: Show Nudity Level of the Body Model

The virtual fitting room provides for a setting, which sets what clothesshould be shown on the model when none of the garments in the fittingroom have been selected. In one example the user can set this and inanother example this is a setting provided for a group or individualusers from the administrators.

On group level this setting can be based on the age of the user or thesize of the user. For instance if the body model has reached a thresholdit is to be shown in underwear or a more covering garment. The settingcan also be based on the location of the user, for instance on countrylevel and this value can be retrieved from the values the user enterswhen registering or automatically based on for example the internetprotocol (IP) address of the user.

When sharing a look the user decides upon how much clothing the modelwears. In order to minimize the risk of improper use the recipient ofthe look can't undress the model. In one example the model is sharedwith the possibility for the recipient to exchange items worn but notdecrease the amount of clothing or undress the model.

Section 14: Entering Values on Sliding Scale Updates the Body Model Live

In one example, when the user is entering the body measurements using asliding scale, the body model shown is changing properties to match theuser's input. This can alternatively be done with the user enters valuesin numbers. The user will then be able to try different garments on tothe model.

An alternative start point is to hide the body model from the user untilshe has entered height and weight. FIG. 19 shows an alternative way ofentering measurements, in a system where hips and waist measurements arealso entered.

FIG. 20 shows an example of alternative dimensions to measure and/or letthe user enter for the creation of the body model.

Section 15: Automatically Adapting the Virtual Body Model to DifferentHeel Heights

The body model can be displayed wearing shoes of different heel heightand the choice of footwear changes the heel height of the body model.The adjustment of the heel height depends on the type and model of thefootwear used. In one example the heel height adjustment is done byadjusting the heel on the body model and pivoting the feet. In oneexample also the posture of the body model will change when the bodymodel has a shoe with a different heel height. The garments the bodymodel is wearing will also be adjusted in accordance with the heightadjustment of the model to account for the extra height because of theraised heel. When the body model does not wear shoes the body model isdisplayed with realistic, flat feet.

Section 16: Automatically Adjusting Hem Length and Body Model Height

The hem length can also be adjusted to account for a shorter or longerbody model. For instance the hem length would appear shorter on a tallermodel. This is because body models of different heights could fit withinthe same garment size. The appearance of the hem length is then adjustedto match with the natural appearance. The actual length of the garmentis kept.

The same problem can occur if the dress for instance is covering thebody model's feet and perhaps the size of the dress is longer than downto the floor. This is in one example to be communicated to the user andcan be presented also with how much it needs to be taken up with to meetcertain criteria. The criteria can be not to touch the ground, to beover the feet or for instance to meet the length of the garment on astandard sized model.

In one example the user is just notified that the dress will touch theground and one option is to see by how much. In another example the useris notified that the dress will touch the ground and given the option toshow it taken up. The user can be presented with the option to specifyhow much to take up the dress by. The user can also or alternatively bepresented with just the option to take the dress up. By selecting any ofthose options, the dress hem length will appear shorter.

Section 17: Changing Hairstyle on the Virtual Body Model

The virtual fitting room allows the user to change the hairstyle of thebody model. In the interface in a separate section the availablehairstyles are shown. The user can then select one of those hairstylesto use together with the representation of the user's head.

This feature allows the user to try different hairstyles together withdifferent outfits. It could be that the user would like to wear the hairup or that she is to try out a new hairstyle before going to thehairdressers. One implementation of this can be seen in FIG. 40 wherethe user can change the hairstyle. FIG. 16 shows how differenthairstyles can be used on one face.

Section 18: Using Different Backgrounds for the Virtual Body Model

As part of the overall look and visualisation of the body model and thegarments the user can choose between different backgrounds. One exampleof this is shown in FIG. 39 where the body model is displayed against abackground and the user has the option to select any of a plurality ofbackgrounds (e.g. colours, images, patterns, photographs etc).

Section 19: Lighting Effects on the Virtual Body Model

The human eye is sensitive to lighting defects and anomalies; as such itis important that the light of the garment is consistent with the lightin the background.

In one example the garments are photographed using several lightsources. The garment is photographed using one of the light sources at atime. The set of images taken using the different light sources can beblended to give the impression of a specific lighting condition of thegarment. This can for instance be used to match the lighting conditionin the background and a different blend of the garment images is appliedwhen the user changes the background. The light temperature of thevisualisation of the body model can also be adjusted to simulatedifferent lighting settings such as light temperature, sunset etc.

Section 20: Automatic Effects Depending on Garments

In one example if the user selects to dress the body model in araincoat, the visualization of the look will include effects of rain.This can be a rainy background or for instance a layer of rain overlaidon the image. Similarly, if the user selects to dress the body model ina pair of swim shorts or a bikini, the background is changed to displaya beach and the lighting is changed to simulate sunlight.

Section 21: Photo Effects Applied to the Garment on a Virtual Body Model

The images where the body model in an outfit is shown can be adjustedwith different effects to change the perception of the look. The effectscan be shown in the virtual fitting room and also be shared as part ofthe shared look. There can be different kinds of effects the user canselect from. The effects can be photographic effects such as out offocus, changed focus, flare; location specific effects such as rain,snow; other types of graphical effects such as dividing the image intiles or giving the impression that it has been painted withwatercolours.

Section 22: Connecting with Other Sites

The virtual fitting room can connect to different types of platformssuch as social networking sites as well as different retailers andsearch engines. The body model can be interacted with from differentplatforms where the user can try on different garments and also shop forgarments. The user can also create looks via other sites.

The garments tried on the body model and the looks created can be sharedand also stored in the virtual fitting room. Connecting the virtualfitting room to other sites enables the virtual fitting room to providefitting and size data to be used by the other sites. The data can befiltered before provision to the partner sites: for instance, onlyproviding anonymised data or providing only user-specific data. Theusers can in one example decide what data should be shared with thirdparty sites.

Section 23: The User's Wardrobe

The user can save clothes to the user profile and store those in a“wardrobe”. These could be clothes the user already owns that she wouldlike to combine with other new garments. It could for instance also begarments the user would like to own and therefore would like to keep forfuture access.

Garments in the user's wardrobe that the user owns could in one examplebe prevented from being deleted centrally from the system and not beremoved from the user's wardrobe even though the garments might not beshown in the virtual fitting room for other users. The user could alsolet other users use clothes from their wardrobe, either to dress thewardrobe's user's body model or for their own body model.

Section 24: Level of Body Model Completion

The user can in an example receive guidance on how far she is increating the body model. Providing the user with this information is oneway to encourage the user to complete the whole process of creating thebody model and update the user profile.

Section 25: Randomized Outfits

The user can be presented with the option to randomize a selection ofgarments on to the body model. The garments from which to choose can bedefined with for instance the garments in the user's wardrobe, garmentsfrom a specific brand, garments of a specific colour or other factors orcombinations of these.

There can be a feature where the user is asked questions before creatingthe random outfit to make it more appropriate for the occasion.

The randomizer can create an outfit for a specific occasion and in oneexample the user can provide feedback to the randomizer engine about theappropriateness of that outfit for that occasion. This could forinstance be for a gala dinner, some sports activity or a date.

Section 26: Informed Randomized Outfits

The selection of garments for the body model can be totally random or itcan be informed. One way the random selection can be informed is aboutthe user's previous buying pattern and for example suggest similar ordissimilar garments, depending on the setting.

One alternative information base the random selection can use is inputfrom a particular store or several stores, depending on a platform'sconnections to stores. A store could feed in to the model that theywould like to push a specific garment. This could be because the marginis better on that one, that the garment is soon to be replaced by nextseason's collection or for instance since the store would like to sellmore of that garment.

See also Section 29.

Section 27: User Satisfaction Survey

One feature that is present in one example of the virtual fitting roomis a user satisfaction survey. The user is then prompted to rate howsatisfied she is with the body model as it has been created.

If the user answers that the model is not accurate then the user will beprompted with a message asking the user to annotate on the body modelimage where it is not accurate. If the user for instance has notprovided a waist measurement and the user suggests that the waist is notaccurately represented she is prompted with a message suggesting thatshe should give that value. There could also be a direct link or inputarea for the user to put in that value.

In an example, the virtual fitting room is provided with anadministration interface.

Section 28: Key Features of Development and/or Demonstration Systems

There is provided a method of visualising garments on a virtual bodymodel, comprising the steps of (a) storing digital images of multiplegarments; (b) storing a virtual body model of a user; (c) displaying thevirtual body model on a touch screen device, together with an image ofone or more garments; (d) combining the virtual body model with theimage of at least one garment; (e) using a touch action or gesture tothe touch screen to cause a different garment to be automaticallycombined with the virtual body model in place of the previouslydisplayed garment.

The method may be one in which a sequence or array of multiple garmentsis shown together with the virtual body model. The method may be one inwhich the garment array is a linear array and the user can with a flickto the left or to the right or other gesture cause the array of garmentsto move to the left or to the right, with the particular garment thatwould occupy a given position behind or over the virtual body modelbeing automatically combined with the virtual body model. The method maybe one in which a drag and drop action is used to cause a specificgarment to be dragged to and dropped on and hence combined with thevirtual body model. The method may be one in which physics-basedanimations are applied to the moving garments.

There is also provided a method for enabling a user to estimate how wella garment, available on a peer-to-peer web site, will fit that user, inwhich a seller of the garment is associated in a database with a virtualbody model defined by various body size measurements; and the user isalso associated in a database with a virtual body model defined byvarious body size measurements, and a system accesses the or eachdatabase to enable the user to estimate how well the garment will fit byenabling a comparison between the seller's and the user's own body sizemeasurements.

The method may be one in which the user manually compares the body sizemeasurements. The method may be one in which the system compares bodysize measurements of the seller and the user that correspond to the sametype of measurement, such as height, weight, size of waist, size ofhips, size of chest, bra size, height to crotch. The method may be onein which the system generates an index or score of how close the twovirtual body models are so that the user can estimate how well thegarment will fit. The method may be one in which the seller does nothave his/her own virtual body model, but such a virtual body modeland/or the related body size measurements can be inferred from otherpersons that do have a virtual body model and have purchased the samegarment. The method may be one in which the web site is a peer-to peerselling service such as an auction site. The method may be one in whichthe web site provides a clothes rental service.

There is also provided a method for generating an image of a first userin a garment, in which the system can access a virtual body model of thefirst user and a database of garment images and a history or record ofprevious garments purchased or viewed by other users, each with similarvirtual body models to the first user, and the system automatically,without user intervention, selects a garment which the system determinesis a suitable garment given the history or record of previous garmentspurchased or viewed by the other users and then combines an image ofthat garment onto the virtual body model of the first user andautomatically without user intervention provides to the first user animage of the garment, combined with the virtual body model, to enablethat first user to visualise what the garment will look like when wornby them.

There is also provided a method for generating an image of a user in agarment, in which the system can access a virtual body model of the userand a database of garment images and a history or record of previousgarments purchased or viewed by the user, and the system automatically,without user intervention, selects a garment which the system determinesis a suitable garment given the history or record of previous garmentspurchased or viewed by the user and then combines an image of thatgarment onto the virtual body model and automatically without userintervention provides to the user an image of the garment, combined withthe virtual body model, to enable that person to visualise what thegarment will look like when worn by them.

The method may be one in which the garment combined with the virtualbody model is displayed on a garment retail web site that the user haslogged on-to. The method may be one in which the similarity of thevirtual body models of the other users to the first user, is a functionof various body size and/or shape measurements. The method may be one inwhich the similarity of the virtual body models of the other users tothe first user is also a function of age. The method may be one in whichthe similarity of the virtual body models of the other users to thefirst user is also a function of user location, user preferences or anyother user-related criteria.

There is also provided a method for generating an image of a user in agarment, in which a system can access a virtual body model of the userand a database of garment images and can automatically, without userintervention, select a garment and then combine an image of that garmentonto the virtual body model and finally, automatically and without userintervention or instruction, send or otherwise provide to the user, at atime that has not been selected or prior approved by the user, an imageof the garment, combined with the virtual body model, to enable thatperson to visualize what the garment will look like when worn by them.

The method may be one in which the system has access to a database ofimages of many different garments and selects which garment to choose byapplying criteria based on factors including one or more of: age of theuser, size and/or shape of the user, location of the user, income of theuser, hobbies of the user, purchasing history of the user, browsinghistory of the user, preferences stated by the user. The method may beone in which the garment combined with the virtual body model is sent asan e-mail to the user. The method may be one in which the garmentcombined with the virtual body model is displayed on a garment retailweb site that the user has logged on-to.

There is also provided a method for generating an image of a user in agarment, in which the system can access a virtual body model of the userand a database of garments or garment images and the user is able toselect from one of several different recommendation channels, such that,when a specific channel is selected, then the system automatically,without user intervention, selects a garment which the system determinesis suitable given parameters of the virtual body model and then combinesan image of that garment onto the virtual body model and automaticallywithout user intervention provides to the user an image of the garment,combined with the virtual body model, to enable that person to visualisewhat the garment will look like when worn by them.

The method may be one in which each channel is associated with aspecific sub-set of images of garments from the complete database ofgarment images. The method may be one in which each different channel isassociated with a different person. The method may be one in which eachdifferent channel is associated with a different stylist. The method maybe one in which each different channel is associated with a differentprint publication. The method may be one in which each garment ismanually scored as being suitable or not for each of a range ofdifferent body shapes and/or sizes and that data stored in a table whichis then automatically accessed and used to enable the system todetermine which garments are suitable for a user given the parameters ofthe virtual body model of that user. The method may be one in which thegarment combined with the virtual body model is displayed on a garmentretail web site that the user has logged-on to. The method may be one inwhich the parameters of the virtual body model include one or more ofthe size or shape of the virtual body model, the age of the user, anyother criteria input by the user.

There is also provided a method of manufacturing a garment, in which auser has a virtual body model of themselves, the method includes thesteps of (a) the user selecting a garment from an on-screen library ofvirtual garments; (b) a processing system automatically generating animage of the garment combined onto the virtual body model, the garmentbeing sized automatically to be a correct fit; (c) the processing systemgenerating data defining how a physical version of that garment would besized to provide that correct fit; (d) the system providing that data toa garment manufacturer to enable the manufacturer to make a garment thatfits the user.

The method may be one in which the virtual body model is generated froma digital photograph of the body of the user. The method may be one inwhich the virtual body model is generated using the height and weight ofthe user. The method may be one in which the virtual body model isgenerated using one or more of the following: width of chest, height ofcrotch etc. The method may be one in which a user takes, or has takenfor them, a single full length photograph of themselves which is thenprocessed by a computer system that presents a virtual body model basedon that photograph, together with markers whose position the user canadjust, the markers corresponding to some or all of the following: topof the head, bottom of heels, crotch height, width of waist, width ofhips, width of chest. The method may be one where the user entersheight, weight and, optionally, bra size. The method may be onecomprising a computer system which then generates an accurate 3D virtualbody model and displays that 3D virtual body model on screen. The methodmay be one where a computer system is a back-end server.

The method may be one where the user takes photograph on a computingdevice such as a mobile telephone and uploads that photograph to aback-end server. The method may be one where the garment is sizedautomatically to be a correct fit without using any standard, basesizes.

There is also provided a method for automatically generating hairstylerecommendations, in which a system receives as input one or morephotographs of a user's face and then (a) analyses that face for facialgeometry and (b) matches the facial geometry to a library of hairstyles,each hairstyle being previously indexed as suitable for one or morefacial geometries, and (c) selects one or more optimally matchinghairstyles and (d) outputs an image of that optimally matched hairstyleto the user.

The method may be one in which the system combines the image of theoptimally matched hairstyle onto an image of the user's face. The methodmay be one in which the user selects a garment and the system selects ahairstyle that is optimally matched to that garment. The method may beone in which the user can select a specific hair colour and the image ofthe hairstyle is altered to use that specific hair colour.

There is also provided a method for automatically generating hairstylerecommendations, in which a system receives as input one or morephotographs of a user's face and the user's selection or choice ofgarment and the system then (a) matches the garment to a library ofhairstyles, each hairstyle being previously indexed as suitable for oneor more specific garments or types of garment, and (b) selects one ormore optimally matching hairstyles and (c) outputs an image of thatoptimally matched hairstyle to the user.

The method may be one in which the system also generates and outputs animage of the optimally matched hairstyle onto an image of the user'sface and a virtual body model of the user combined with the garment.

There is also provided a method for automatically generating make-uprecommendations, in which a system stores a virtual body model for theuser, together with one or more garment selections made by the user anda library of make-up descriptions; in which the system automaticallydetermines appropriate make-up descriptions given the specific garmentor garments selected by the user and then provides an image of thevirtual body model combined with one or more of the garments, plus themake-up description.

The method may be one in which the make-up description is make-upadvice. The method may be one in which the make-up description is animage of make-up applied to an image of the user's face. The method maybe one in which the system stores the skin tone of the user; and themake-up description the system provides is compatible with the skintone.

There is also provided a method in which the system receives as inputone or more photographs of a user's face and then (a) analyses that facefor facial geometry and (b) matches the facial geometry to a library ofmake-up descriptions, each make-up description being previously indexedas suitable for one or more facial geometries, and (c) selects one ormore optimally matching make-up descriptions and (d) outputs an image ofthat optimally matched make-up description to the user.

The method may be one in which the automatic determination is achievedby the system looking up in a database appropriate make-up descriptionsthat have previously been selected by make-up experts, stylists, ormodels for specific garments or categories of garments.

There is also provided a method of enabling a viewer of a televisiondevice to visualise what a garment, displayed on a program shown on thetelevision device, would look like if worn by the viewer, comprising thesteps of (a) processing a virtual body model; (b) providing an image ofa garment on a second, portable display device together with an icon,button or function that, if selected by the viewer, automatically causesone or more images of the garment to be combined onto the virtual bodymodel to enable the viewer to visualise on the second, portable displaydevice what the garment will look like when worn by them.

The method may be one where the second portable display device is amobile telephone or tablet. The method may be one in which the programis a live television program and feedback is provided from the portabledisplay devices such that the program includes an indication of thenumbers of viewers visualising a given garment on their virtual bodymodels. The method may be one in which the second portable displaydevice includes a button, icon or function that, if selected by aviewer, automatically sends a signal from the device to a server toindicate that the viewer wishes to purchase the garment. The method maybe one in which the signal authorises automatic payment for the garmentby the viewer.

Section 29: Recommendation Features

Any recommendation from a retailer may be influenced by the businessgoals with wanting to ship high margin garments or garments that theyhave a big stock of.

Retailers can in one example enter business rationales for the virtualfitting room layout. Stock levels and margins could be part of how thegarments are laid out in the fitting room and what is recommended forthat particular user. Business rules having an impact on therecommendations can be margin, stock levels and also any chosendiscretionary rules. Magazines may give advice more freely.

Recommendations can be given in relation to the natural fit, which isthe fit of garments for a specific body type. The recommendations can begiven via the virtual fitting room to the users to use for their bodymodels. The recommendations can in an example only be shown to userswith the matching body size. The recommendation can alternatively beshown as an aspirational recommendation where users receive them to havesomething to strive towards.

See also Section 26 and 83.

Section 30: ‘Cover Flow’ Type UX for Garments

The virtual fitting room provides for a fast and easy way for a user tobrowse through a large set of different garments, similar to album art‘cover flow’ used in many online music services. The way the user canflick or swipe through the set of garments, for instance using a touchscreen device, is exemplified in FIG. 46.

The user can select what type of garment they would like to flickthrough to try out on the body model and also select one of the garmentsto stay on the body model. The user can also select to flick throughwhole outfits of garments to be tried on to the body model. The flow ofgarments can be flicked through in any horizontal direction fordifferent garments.

In one example the browsing tool allows the user to flick vertically tochange the set of garments to flick through. It can be understood asseveral different horizontal rows of garments that the user can alterbetween.

The garment pane can be slowed down and a garment selected and the bodymodel as displayed is then dressed in that garment. The user can selectto continue to flick through alternative garments to replace theselected garment or the user can select to flick through alternativegarments to be worn together with the selected garment.

The user can also select and filter depending on their location. Theycan filter on the location or if they have selected a garment the usercan see where the closest location supplying that garment would be.

The user can also get specific targeted adverts based on the garmentsthey are viewing and be provided with that information from the retaileror third parties. The adverts can also be linked to location and can forinstance be that the garment you are viewing is 10% off if you enterthis close-by shop within 30 minutes.

Section 31: Automatic Suggestion of Outfit

In an example, there is provided a feature where you take an image of anoutfit worn by a person and the device or service uses that image andmatches it (or the corresponding outfit) to the selection of garmentsthe virtual fitting room has in its database.

The search can try to provide an absolute match of the garments andsuggest the same ones as seen in the picture. The search canalternatively try to provide a best match to the garments identified inthe picture. That best match can be a best match based on colour,material and shape, depending on what the user prefers. The best fitmatch can provide an interface where the user can decide on what featureof colour, material and shape should be the highest priority. A weightrange can be provided; this could be by entering values or for instanceby moving a slider to a place on a scale.

Section 32: Virtual Fitting Room

In the virtual fitting room, the retailer's customers can have anenhanced experience using the body model.

The three main benefits to the retailers are:

1) Conversion Rate

The people who create a body model are more likely to purchase from theonline store. The virtual fitting room has a positive impact on theonline retailer's conversion rate (the rate of converting viewingcustomers to buying customers).

There is a strong suspicion that the “fast fashion” is the demographicthat is the most acceptable to the online retailing experience. Thatgroup will not have as strong views and concerns about data privacy andwill also be more open to share looks with other users.

2) Basket Size

If a user shops for garments in isolation they tend to focus on onegarment at a time. The virtual fitting room allows the users to shop fora whole outfit and this is driving the basket size. The users will addmore garments to the shopping experience and also have a more playfulattitude to shopping.

3) Positive Impact on Return Rate

The virtual fitting room reduces the return rate for garments, since theusers already have tried out the garments in the fitting room beforedispatch to their homes. Returned garments are costly to retailers bothin the logistics in handling them and also the diminished value from areturned garment.

Section 33: Cross Promotion of Goods

The users can be presented with recommendations for products innon-garment sectors. For example if the system calculates that a personbased on the body model size is overweight, that information can feed into the retailer's system recommending diet products.

Some retailers that today have club card systems lack size and fit datain relation to the cardholders. The lack of that data in the systemlimits what products and services they can offer through their club carddatabase and related services. Connecting the data and buying behaviourfrom the virtual fitting room adds a whole new dimension to the systems.The retailers can create loyalty programs around size and fit and theycan provide information in relation to what other shopping behavioursthe users have. The virtual fitting room can also be connected to thatdata and provide that information as well as provide shoppingrecommendations based on that data.

Section 34: Body Size Matching and Indexing for Improved Peer to PeerClothing Purchasing BACKGROUND

By way of example, Metail solves the problem of online clothing fit byallowing users to create 3D photo-real body models of themselves. Metailalso then digitizes garments in its special digitization rig to allow itto match garments to bodies for fit visualization. However there aremany instances where Metail will not be able to obtain garments fordigitization to enable it to generate fit visualization, such as for thesecond hand market. In this instance an alternative solution is neededto give users the confidence to know that purchases may or may not fitthem.

Problem

We solve the problem of allowing consumers purchasing or rentinggarments from other users to get an indication of what garments are mostlikely to fit, by getting an understanding of the body match of sellers.This should give an increased confidence of buyers from sellers wheresellers are selling their own previously worn items.

DESCRIPTION

A method for scoring garments based on fit. As we are building adatabase of users who will be creating ‘Body models’, through Metail,i.e. photo-real accurate virtual body representations of themselvesonline for trying clothes on to see how they fit, we will therefore beable to match users to other users who have a similar body shapeprofile. This information will give users an indication of whichgarments being sold by other users through channels such as eBay mighthave a better chance of fitting them. We will provide an index matchingsystem such that a user could search for items from people with a 95%body match index or they could even broaden the search to 80% forexample. The body match index will be built based on all the fit matrixdata that Metail generates from its users.

An advantage is not having to generate garment visualization images,because body data and algorithms are used instead. Therefore there willbe no garment digitization costs and logistics. A further advantage isincreased basket size and user purchasing potential for second hand andrental businesses through increased confidence to purchase.

Examples

A specific example is a body matching system for eBay. Another specificexample is for a clothes rental service.

Other Ideas

The idea of crowd sourced or peer-to-peer fit recommendations forgarments that the Metail computer-implemented system cannot model.

Initially thought of for clothes rental but this can be used for avariety of services such as second hand and out of production garments.

Could work with the likes of Ebay, since the service provider has notseen the garment before a user wants to sell, but the user can rate howwell the garment fits and that recommendation can be a guide for aperson with a similar body size and shape.

One way you could do peer-to-peer selling of garments is that Metail cantell you that “you are a close match” to a seller.

What else could you do with crowd-sourced fit recommendations?

Ripe for game-ification:

Stack overflow model: the more you recommend the better offers you get.

Ask questions to the community and receive answers to get feedback. Gethigher in the hierarchy level.

Other:

Interaction around the Me Model, stardoll model. The models should existstand alone on a platform, webtop, online. Share and collaborate on themodels.

Generally the idea that information about how garments fit some userscan be used to infer how well they might fit others. We can match usersto other users with a similar body shape profile, allowing e.g. clotheson eBay to be reliably bought if the buyer and seller have a sufficientmatch in their body shape profile.

Linked Garments

One useful feature, which the virtual fitting room provides for, is toaggregate data on which garments are bought together.

The garments shown as being bought together could be bought by the sameperson over time or they could be bought by the same user at one time.

The linkage can also be among friends where information can be providedabout what type of garments and what particular garments friends buythat are similar. Similarity can be in the types of specific garment orin the dress colour for instance.

The friend connection can be derived from the users being friends(having accounts that are linked) on either the virtual fitting roomplatform or in a different social networking site such as Facebook.

The information about similar shopping behaviours around garments canalso be for similar body sizes. For instance it can be derived thatpeople with a height between 150 cm and 165 cm more often buy acombination of a dress and boots than people of other heights.

Section 35: Using Information about Linked Garments

The use of the data collected can be for recommending users to try thelinked garments. It could for instance be that your friend has bought adress and you get a recommendation to try the same or a similar dress.Or that someone with your body size has bought a particular shirt andyou are recommended to try that one on.

The user could also get a recommendation that one of the garments theyare trying out has complementary garments. The complementary garmentscould be from previous shopping behaviour of other users.

It could also be data that is being provided from the retailer, or forinstance from a stylist, that some garments go well together. Therecommendations can also be crowd sourced from the user base and theycould come with recommendations on what fits together for instance. Seemore about recommendations elsewhere in this document.

Section 36: In-Store Concepts

The virtual fitting room can be connected to an ecosystem which in oneexample includes physical presence in retail stores. An outline of anexample system is shown in FIG. 2-9.

The ecosystem is one in which the customer can use an online body modelor can go in to a real-world retail store and create a body model ofherself though a picture and by entering measurements or otherparameters. The user can see the body model and try on clothes using anin store computer and after the experience also access it online. Theuser can also access the body model through her mobile phone where shefor instance can scan a bar code of clothes and see them on the bodymodel.

A virtual fitting room service for online clothing retailers andmedia/fashion websites is provided. An outline of the system is givenbelow:

1. Concept

a. Consumers create 3D ‘Body model’ versions of themselves in-store

2. Scope

a. Big established retailer

b. Potential to go wider

3. Audience

a. Users=tourists=> local customers post Olympics in London 2012

4. Multi-channel

a. Profile for use online, in-store and on mobile

5. Aim

a. Social interaction driving brand, PR and sales through lifetimeengagement with customer

The in-store concept is part of an overall system which consists ofseveral parts that all centre around the virtual fitting room and thebody models.

In-Store

create my own ‘Body model’

get all my body measurements

try lots of outfits without needing the changing room

create my own style magazine

enter cool new style competitions

create my own ‘Body model’

get hair & makeup advice from style advisors stored to profile

try more trends, faster in-store on Body model in Style Surgery

match hair & makeup to clothes buying

Mobile

scan items/marketing campaigns and try them instantly on the move

Online

use my profile to continue trying clothes on at home

see what others think about my style choices

use my profile to try different makeup and hairstyles on at home

share and see what others think about my style choices

match to clothes and total ‘look’

Technology Features

Create virtual body model, try on and save outfits

Interact with model in-store through various means:

Changing room booth (private)

Small touchscreen (less private)

Large touchscreens/plasmas (public)

Big screens can show two models at the same time so that friends canshop together and dress each other

The smartphone application allows the user to try outfits on their modelin or out of stores (barcodes could be scanned)

Microsoft Kinect allows for gesture recognition and can be used withlarge plasmas instead of touchscreens

Social Features in Connection with Store Concept

The user can publish her looks to a slide show on a touch screen in thePartner store/an advert (this can be ‘liked’ leading to prizes)

Look Books can display the most ‘liked’ looks on a large touchscreen

Most ‘liked’ looks can be compared between stores (long term)

Publish your looks to the Partner website

Publish your looks to Facebook and other social media sites

A celebrity look can be viewed or incorporated

Styling advice can be given based on skin tone, size and shape

Style surgery and advisors in-store

A ‘Look Off’ game can be created to allow users to try to create thebest look (could be judged by a celebrity)

Print Features

Your looks onto a photo strip (could include randomised looks)

Your measurements and recommended Partner size

Your images to create postcards or save as e-cards (themes)

Your images to create your ‘Partner Magazine’

Loyalty

Loyalty card integration

A picture of your model can be printed onto your card

The card can be swiped in store or code entered online to track the userand update Metail wardrobe

Given a store and online loyalty card when you create a model (tracksize over time and adapt offer)

In-Store and Online Excitement

1. Save size and outfits to the Metail plug in on the Partner website

2. Differing levels of privacy, from a changing room booth, to a smalltouch screen to a large touch screen

3. Publish your looks to a slide show on a touch screen in the Partnerstore/an advert (could be ‘liked’ leading to prizes)

4. Look Books can display the most ‘liked’ looks on a large touchscreen

5. Most ‘liked’ looks can be compared between stores (long term)

6. Publish your looks to the Metail website

7. Publish your looks to Facebook and other social media sites

8. A celebrity look can be viewed or incorporated

9. Your looks onto a photo strip (could include randomised looks)

10. Your measurements and recommended Partner size

11. Your images to create postcards or save as e-cards (themes)

Mobile and Additional Features

A Smartphone application allows the user to try outfits on their modelin or out of stores (barcodes could be scanned)

The screen can show two models at the same time so that friends can shoptogether and dress each other

Styling advice can be given based on skin tone, size and shape

Your images to create your ‘Partner Magazine’

Loyalty Again

For instance the Microsoft Kinect allows for gesture recognition and canbe used with giant plasmas instead of touchscreens

A ‘Look Off’ game can be created to allow users to try to create thebest look (could be judged by a celebrity)

Loyalty card integration

A picture of your model can be printed onto your card

The card can be swiped in store or code entered online to track the userand update Metail wardrobe

Given a store and online loyalty card when you create a model (tracksize over time and adapt offer)

Creating the Model

An example is shown in FIG. 4.

The consumer takes a photo of themselves in front and side profile in aphoto booth.

Their body is created and an online profile is created ready for them totry on clothes.

All products viewed on that day are automatically saved to their profileon the Partner website meaning that users can review (and purchase) theitems they saw that day at any time.

Scan Garments in Store

Customer uses Partner application on their mobile device to capture theproduct barcode in-store.

An example shown in FIG. 5.

Server identifies product and presents product detail to the customer.An example is shown in FIG. 6.

The customer selects the product and then can immediately view thegarment on their own personal body model in full 3D. An example is shownin FIG. 7.

Share the outfit with close friends for immediate feedback on Facebook.An example is shown in FIG. 8.

. . . or if the customer is bold she can publish to the website orin-store ‘Style Wall’ to see if she can get the most ‘liked’ outfit ofthe month. An example is shown in FIG. 9.

What are the Key Differences Over Known Systems?

-   -   Systems today are all based on virtual mirror systems in-store,        which revolve either around        -   taking photos of you as you turnaround to then present them            back to you and share with a community or;        -   an augmented reality approach or increasing the size of            clothing image on screen to match a size to place over a            video image of yourself as you stand there. No precision but            merely a visual cue and simple technology wrapped up in the            exciting sounding words of ‘augmented reality’    -   Our technology revolves around you creating your Body model        in-store and then ‘trying’ the clothes on that 3D Body model        either on small touchscreens or in-store computers or even on        full length life-size screens controlled either by touch or        gesture recognition systems like the Microsoft Kinect

Other Aspects of the in-Store Ecosystem

Creation of physical body sizing chart printout from photos taken, i.e.we will know your size and shape dimensions and likely size across therange of clothing presented in-store, so could give each customer whogets their photo taken a print-out of their dimensions and sizes in thevarious clothing types sold—simple yet highly effective.

Creation of Body model magazine or post cards that users could purchase,i.e. from a Body model created, the retailer creates a glossy printoutmagazine for the customer to purchase or give-away free in-store, i.e.the user as the star of their very own magazine dressed in all theseason's top trends.

The screen can show two models at the same time so that friends can shoptogether and dress each other.

A ‘Look Off’ game can be created to allow users to try to create thebest look (could be judged by a celebrity).

Loyalty card integration. The card can be swiped in store or codeentered online to track the user and update Metail wardrobe. Given astore and online loyalty card when you create a model (track size overtime and adapt offer)

A Smartphone application allows the user to try outfits on their modelin or out of stores (barcodes could be scanned). An example of this isshown in FIG. 5-7. Publish your looks to a slide show on a touch screenin the Partner store/an advert (could be ‘liked’ leading to prizes).

Section 37: Personalized Fashion Tips Based on Your Body Size

Personalized fashion tips can be provided based on your body size andmeasurements. The tips could also be customized to match the features ofthe garments used in the clothes. The data is gathered from theinformation you have entered and also from what you previously havebought. The styling tips can be done in collaboration, with for instancefashion magazines.

Personalized clothing tips for specific clothes based on your body sizeand measurements. Get recommendations for clothes and sizes for specificretailers or brands. The data is gathered from the information you haveentered and also from what you previously have bought.

Section 38: Style Radio

A push recommendations system in which designers give custom advice tocustomers—e.g. ‘for your size and shape, this shirt works best etc.’Users can ‘dial in’ to their favourite stylists for their subjectiveadvice overlay over our objective initial work.

From our perspective, building the system is such that the virtualfitting room service has the subjective measures and then each stylistadds their idea for garment type and brand per body shape and we buildthis into a database for what then is shown as dynamic advice.

Section 39: Maternity Clothing Feature

The virtual fitting room provides a maternity clothing feature where theuser can input measurements and the progress of the pregnancy and thebody model is modelled to reflect that. The body model can also beadjusted to reflect a future state in the pregnancy. The user can alsoget recommendations and purchase clothes that will fit when reachingcertain state of the pregnancy.

A manual adjustment can be made also for height, for instance tosimulate the growth of a child. See also the predictive feature forgrowth described separately.

Section 40: Compare Two Body Models with Different Outfits

The system allows for a user to compare body models of herself indifferent clothes to try out different outfits. The system also allowsfor the user to see the body model of a different user to for instancesee how the outfits of the two users match.

Section 41: Direct Marketing Including a Picture of Your Body Model inSelected Clothes

The system allows for the system operator or a third party to addclothes to a user's body model and for instance send it as an email tothe user. This could be done via the system without a third partygetting access to the user's model or measurements.

It could for instance be if the retailer is launching a new collectionand sending out an advert showing your body model dressed in garmentsfrom that new collection. The promotion can also be based on yourprevious shopping preferences.

Retailer could send out effectively marketing magazines personalised tothe user with the user in every shot or if aspirational, a model closestto the user's size and shape in every shot. Other NON-CLOTHING retailerscan also use this system, i.e. here is you with this lovely ice-cream.This is personalised direct marketing because we will own the userinformation. This non-clothing part will depend on privacy settings butthe way to drive that would be from advertising campaigns pulling theuser into the adverts—driving the ability to continue to use that datain future for these purposes.

Precise marketing using this system could be to only send advertisementsfor garments where only size 10 is in stock to those people having size10 in that store and in that fitting model. This targets the long tail.This could be sending recommendations to people having bought or havinggarments of a particular colour in their virtual wardrobe, providingrecommendations for garments of that colour.

Section 42: Real Avatar

The system allows the user to use the body model in for instance onlinegaming or social applications or networks as an avatar. This would allowthe user to for instance use his/her own face and body shape whenplaying online games. The user would also be able to animate andinteract using his/her own body shape and face.

Using the virtual model in gaming can be seen from two differentaspects 1) one where the body model created for apparel is used ingaming and 2) one where the techniques to create the body model isapplied specifically to create a gaming avatar with the user's body.

Section 43: Bespoke Tailoring on Body Model

The system would enable the user to use the body model for bespoketailoring and try on different materials and clothes and also order thetailor-made clothes from the interface. The system could use basicstandard shapes for the different types of clothes and then only adjustthem to fit the body model of the user. The system could also enable theuser to fully tailor his own clothes.

A separate use for the body model is in online bespoke tailoring ormade-to-order tailoring. This can be enabled through designermarketplaces, which are great for young designers to access peoplewithout having to spend a lot of money on creating samples, i.e. theycan build their own market.

Section 44: Aspirational Modelling

One aspect of the virtual fitting room and the body model presented tothe users is that the impact of a photorealistic model is hard topredict. In one example the virtual fitting room provides tools thatallow the users to dwell in the more conventional aspirational realmfrom time to time—being able to switch between their Body model and apicture of a model of similar stature.

Section 45: Hairstyle Recommendation Engine

Use of facial geometry gained from the single face input image to crossvalidate against hairstylist information about what hairstyles suit whathead shapes to generate recommendations back to users about whathairstyles might suit them. Like style radio, users can dial in to theadvice of celebrity hair stylists, e.g. James Brown (currently in UKtelevision programme Great British Hairdresser) for advice on what to dowith their hair.

Also relate this hair advice directly to clothing choices. Thecombination of recommendation for hair and outfit provides unique valueto the user. With the full Body model setup, i.e. what hairstyle tochoose with that black cocktail dress or green ball gown etc, the userscan share full looks that represent them.

Celebrity hairdresser could give recommendations. The user can viewherself in different hair colours based on the colour set from a hairdye dotcom website. Then match with hair style company. Then match withclothes.

Section 46: Try Out Different Makeup

The user can try out different sets of makeup. Different brands,different makeup looks on their face.

Section 47: Beauty Recommendations

The user can get recommendations for makeup based on their size and/ortheir skin tone and face shape.

Leverage for makeup to skin tone matching: Apparel matching to skintone; Leverage for makeup to skin tone matching.

Section 48: Beauty Recommendation Engine

Use information gained on facial geometry and skin colouration toprovide make-up and beauty advice on what mixture of make-up colourswould work best. Also relate this beauty advice directly to clothingchoices.

The user is able to combine a makeup with clothes and also a haircut orcolour. This is part of the virtual fitting room offering a full Bodymodel setup, i.e. what makeup to wear with that black cocktail dress orgreen ball gown etc.

Section 49: Facial Hair

The male user can try different facial hair styles on the face. The usercan also get recommendations for facial hair in combination withoutfits.

Section 50: Glasses

The user can try on different glasses such as reading glasses andsunglasses with different outfits. This lets the user match an outfitwith a pair of glasses. The user can also get recommendations for whatglasses she should wear.

Section 51: Tattoo

The user can in one example select tattoos to be applied to the bodymodel. A tattoo is treated in the same way as a garment and can bedisplayed and layered. The layer settings decide if a tattoo can bedisplayed at any other level than the inner layer.

Section 52: Two Screen TV

The two screen TV concept is built round user interaction on twoscreens. One screen, the TV, will broadcast a show and the user will beable to on the other screen interact with features relating to what isbeing broadcasted. The interaction screen can be on a laptop, pc,tablet, phone or any other device (typically a portable device) whichallows the user to input information and also receive information inrelation to what is being shown on the TV. In one example, the secondscreen is a smart remote supplied with the TV.

The user will be engaged in the conversation around the broadcasted showat the time when the show is happening. The user can be shown images oroutfits in relation to the program and for instance vote on thedifferent outfits. The users can also comment on the outfits and in somecases what has been interacted upon.

The users can also bet live on events that are occurring on the TVscreen. The odds could be changing in real time and also winners may beannounced in the TV broadcast or on the interactive display. The userscan bet against other users and see the other friends and/or their bodymodels.

The users can also bet on different outfits and also on different lookson different body models. The users can in one example see a real outfiton a person or a mannequin on the TV show and select to dress the user'sbody model in that outfit.

In one example the TV show can show one or several of the user's lookson the TV show, either in real time as the look is being prepared or ina ready state where the user has dressed the body model in the look.

The two screen concept can also be applied if the user is viewing arecorded session. The focus will then be that the user is interactingwith what is being presented on the TV screen and the events as they areunfolding on the TV show.

The two screen TV concept will also be connected to a shoppingexperience where the user also can link in to shopping of the garmentsand other products shown on the TV show.

The two screen concept can be applied to TV shopping channels where theuser can view garments being presented with shopping information and theuser can interact and try the garments on their body models on thesecond screen as they are viewing the programme. The users can also goback and see previous programmes and the garments that have been shownon the previous programmes and also try those garments on the user'sbody model.

Overview Example Launch Concept

Consumers try clothes on 3D models in the BeautyTV styling room.

Scope

Capsule wardrobe of 40+ items with monthly refresh

8 models shot to match the 8 body types currently employed

Users can tweak the models to match their body measurements

Clothing sets chosen to suit the 8 shapes

Users able to try clothing sets on the different body shapes to ‘prove’the styling advice

Co-created content and links to relevant TV programming whereappropriate

Users can create their own ‘Body models’ of themselves direct fromphotos.

TV Show Partner's Style Me

Your Ultimate Personal Fashion Experience.

1. Ease and Convenience

Try the latest fashion trends and styles on yourself in two easy steps,all from your living room!

2. Try the Looks as you See Them

See TV Show Partner's fashion latest High Street trends on yourselfwhile you watch the show

3. Join the Conversation

Share your looks to get votes and advice from friends and the TV ShowPartner Facebook community, and return the favour

4. Feature on the Show

See whose look is the most loved at the end of each show and getinvolved with competitions

Pre-Show Experience

Re-activation and preparation: Morning email; Introduce the theme; StyleMe guide; Teaser content; Preview ads; What's coming up; Making yourmodel is easy; Social Media; Facebook wall post; Announced on Twitter.

The Dannii Minogue celebrity wardrobe is added to Style Me. Viewers cancreate outfits on their body model from the wardrobe and enter them intothe competition. Dannii chooses the best look and the winner getsDannii's Style Me wardrobe in their size.

Post Broadcast Experience

Save your favourite looks in your wardrobe for future reference;Celebrity wardrobe competitions; Expert Q&A and forums; Voting andRating of garments—help influence future selections; Special offers;Community; Integrated with the YouTube channel

TV Show Partner's Style Me

Create The Ultimate Personal Fashion Experience

Enhance TV Show Partner's current position as fashion influencers,making the latest trends even easier to access through enhancedpersonalisation tailored to each viewer

Increase engagement in the already very active TV Show Partner socialmedia community, complementing the TV screen content with the 2nd screenexperience

Build a comprehensive database of information on size, fit and style ofviewer-ship

Section 53: Styling Room for Publishers

Currently styling magazines are engaged in giving recommendations totheir readers but the readers do not have a good way for selecting therecommended outfits and for buying the outfits.

The virtual fitting room concept can provide a valuable channel forpublishers in the form of a styling room where they can show, let thereaders play and try out the recommended clothes and also get part ofthe revenue from the recommended clothes.

The virtual fitting room can provide an environment where the magazinescan provide interactive spaces for the pieces they run in the physicalmagazine with connections to retailers and also the possibility for thereaders to use the outfits and play around with them.

The virtual fitting room is also providing an interaction platform forpublishers and they can let their users create outfits that theretailers can use in their publications or dress real people in.

The publishers can also act as an intermediary who is digitizinggarments. The publishers often receive garments from brands to reviewearly and to include in new trends. This provides for the publishers tobe able to combine clothes from different brands and can also combinehigh end and low end of the brand spectrum.

See also Section 56

Section 54: Ageing Body Model

The body model can in one example be shown in different stages ofageing. This is a useful feature if you as a user are shopping forclothes for your kids and would like to know how well the clothes wouldfit in the future when they for instance have aged one year and alsohave grown accordingly.

The system to calculate the ageing process will in one example take thedata from databases of information about the height and weight ofchildren in different age groups. The data can in another example bederived from the user base in the system and the ages of the users.

The data to calculate how the user would grow over time can also bederived from the size updates the user has done historically in theservice. The calculation from those changes can be linear or use inputsfrom another source to alter variables accordingly.

It is understood that in a preferred example the growth of the bodymodel can be modelled based on a number of different data inputs, eitheron its own or in combination.

In one example the face is being changed to represent the commonfeatures for a person of the relevant age.

Section 55: Give the Body Model a Tan

A useful feature that the user can be presented with is to give themodel a tan. The body model's skin would be slightly coloured when theuser selects that feature. In one example the user is presented with ayes/no switch for applying the tan to the body model's skin tone. Inanother example the user can select how deep the tan should be, eitherin steps or on a sliding scale.

Section 56: Testing of New Looks for Future Collections

Any retailer struggles to understand what their customers would want tobuy when the clothes are in the stores of their shops. The virtualfitting room provides a channel for the retailers and fashion brands totest their garments before placing orders for them or before stockingthe shops.

The retailer could put the garments for the next collection in to thevirtual fitting room and let the users test, rate and share the looks.There could also be a more closed testing where only a select group ofusers were allowed access to the garments to be tested.

In the system there can be provided controls for the retailer to decidewhether the user trying out the new unreleased garment is to be allowedto share the garment with anyone, share the look with anyone or how theycan interact with the garment in some other way.

The retailer would get feedback from the system on the user behaviourwith the garments and if that option has been provided for that garment,how the users rated the garment.

The virtual fitting room is driving the stocking and flow: Section 56:The retailer can understand what garments are being tried out indifferent parts of the country. Also to understand what garments aresold online in the system. In regular stores the manager decides howmany of what size. In more modern stores that is set by a total stockingsystem.

See also Section 53.

Section 57: Key Features Used in Back-End Processing and Photography

There is provided a method of generating a photo-real virtual garment tobe combined onto a virtual body model, in which (a) one or more digitalphotographs are taken of the garment; (b) one or more sprites areextracted from the garment photographs, showing some or all of thegarment and separated from any background; (c) a processing systemmodels the behaviour of the sprite as a virtual garment, applying finiteelement analysis to model how the virtual garment would sit or formitself over a virtual body model of defined shape and/or measurements,when values for variables, including one or more of physical garmentelasticity, weight, density, stiffness, material type, thickness, areinput.

There is also provided a method of generating a virtual body model, inwhich a user takes, or has taken for them, a single full lengthphotograph of themselves which is then processed by a computer systemthat presents a virtual body model based on that photograph, togetherwith markers whose position the user can adjust, the markers

corresponding to each of the following: top of the head, bottom ofheels, crotch height, width of waist, width of hips, width of chest.

The method may be one where the user enters height, weight and,optionally, bra size. The method may be one comprising a computer systemthen generates an accurate 3D virtual body model and displays that 3Dvirtual body model on screen. The method may be one where a computersystem is a back-end server. The method may be one where the user takesa photograph on a computing device such as a mobile telephone anduploads that photograph to a back-end server.

There is also provided a method of developing a standardised sizingtable, scale or chart for garments, including the steps of (a)processing individual virtual body models from a group of users; (b)generating a description of the size and shape of users across variousparameters, including: age, location, any other personal demographic,garment retailer and (c) generating one or more standardised garmentsizing tables, scales or charts using that description.

There is also provided a method of garment segmentation, in which animage of a garment is cut, separated or segmented from the backgroundimage, where the garment is scanned or imaged with depth information andthat depth information is used in a processing system to automaticallydifferentiate pixels in the image that correspond to the garment frompixels in the image that do not correspond to the garment and to thencut the image of the garment from the back-ground and then use thatimage in generating a virtual 3D image of the garment.

The method may be one in which the garment is imaged with depthinformation by using a stereo camera. The method may be one in which thegarment is imaged with depth information by using a depth sensor thatincludes an infrared laser projector combined with a sensor whichcaptures video or still data in 3D. The method may be one in which thevirtual 3D image of the garment is combined with a virtual body model ofa user.

Section 58: Back End Processing

By way of example, eight angles the body model and the garments can beviewed from are shown in FIG. 17-18.

The overall goal is that the customers are to buy clothes they like.This is done through 1) Building a model of the user's body, 2) buildinga model of the garment size and fit and 3) visualizing the garment onthe user's body model.

The visualization can be used to communicate size and fit properties ofa specific garment to the user. The information can be such as how fardown the legs the hemline might come and what size the user most likelywill have with that particular garment. Part of the information thatalso can be conveyed is how tight or baggy the garment will be on thatparticular body shape.

Another way of visualization of garments is to build a 3D model of thegarment, such as by using a stitching plan and visualizing it usingthat. In models like these there can be included fabric specificfeatures such as the fabric's weight and elasticity for instance.

Section 59: The 2.5D Solution

Photographing the garment from a number of different viewpoints on amannequin may be performed. The photographs from several differentviewpoints give the user the impression that the body model is rotatedand it can be experienced as being in 3D. Using this solution to adaptthe two dimensional model of garment shape to be represented on a numberof body models with different shapes is one of the main focus areas forMetail. The approach is to stretch the garment to match the body modelfit of the garment.

This approach has the advantage that there is no need for access to thestitching plan information of the garment. One photograph is needed tovisualize the garment on a body model. The Metail solution with manualpost processing of the pictures of the garments enables the virtualfitting room to visualize outfits and combinations of garments.

See also Section 73.

Section 60: Metail Digitization Process

The garments can be prepared before the digitization process by steaming(eg. to remove creases) or by visual corrections to the actual garment(eg. to remove dangling threads). A mannequin is then dressed with thegarment to be photographed. The photographic setup follows a specifiedlayout where the garment is photographed from eight different angles.The mannequin is rotated to set positions and a photograph is taken; thephotograph is then fed into an annotation system.

The photographic setup also includes a camera where the focal length isfixed at one level and the position is the same for each photographedgarment.

It is important that the mannequin is photographed in the exact positionfor every angle and for every garment. This is because the postprocessing and the visualization of the garment on the mannequin arefrom a specific angle. This is also to enable one garment to becompatible with another garment, especially when combining garments asan outfit and so the garments can be viewed from the same angle when thebody model is dressed in the garments.

Off the shelf mannequins have several joints between the foot and thehead where only a slight misalignment in the photographic process willcreate a significant misalignment of the garment from other garmentsphotographed. Depending on the material of the mannequins they also tendto deform and change shape over time due to ageing and materialdifferences.

In order to keep the joints in exactly the same place on differentmannequins and on the same mannequin from one photo session to the next,one solution is to use mannequins manufactured with detents. (‘Detent’is the term for a method, as well as the actual device, used tomechanically resist or arrest the rotation of eg. a wheel, axle orspindle.)

A different problem when photographing a mannequin on the turntable isthat after each turn the mannequin might wobble slightly. Taking aphotograph when the mannequin is not in a fixed position might give thegarment a slightly different view and position in the photograph. Thisis overcome with a time delay between when the mannequin has turned towhen the photograph is taken. The mannequin is then still in oneposition.

Image Alignment

If the garment photograph is not in perfect alignment with the idealposition of the garment, image warping may be applied using the thinplate spline method. This alignment allows for repositioning andreshaping of the garment in the photograph. With this method the wholegarment or part of the garment can be repositioned.

Section 61: Multi Layer Garments

Many garments have multiple layers that need to be modelled, such as along sleeved shirt with a torso, where the back sleeve and a frontsleeve occupy different depth layers in the scene. This has significanceespecially when the garment is not the same size as the body model.

Layered garments will also change their position when the body model ischanged in size. For example, the arm in relation to the torso may needto be slightly tilted outwards or backwards when the body model isfatter.

Partitioning the garment in depth layers is also important for thecombination of outfits. For example if combining a jacket and a shirt,it is important that the back sleeve of the shirt is behind the torso ofthe jacket.

This can be solved through taking different photographs to representdifferent depth layers of the garment. For example, for a pair oftrousers different photographs may be of a front leg and a back leg; fora shirt different photographs may be of a front sleeve, a back sleeveand a torso; for a shoe, different photographs may be of a front viewand a back view.

Section 62: Deforming Mannequin

A deforming mannequin may be used to model what a garment would looklike if the garment would not have the best fit for a specific bodymodel. Capturing also what a garment would look like if the fit on thebody model were not the best. The user can select alternative views oralternative sizes for the user's body model to show what a too large ora too tight garment would look like.

Capturing the nuances of a garment when having different fits todifferent body shapes is important for the visual appearance of thegarment on the body model. A loose hanging garment might have verticalcreases and a really tight garment might have horizontal creases.Nuances like that are captured when for instance using a deformingmannequin.

Similar results can be achieved when using a mannequin of one size andgarments of different sizes, when the mannequin is dressed in differentgarments for a photo session.

Section 63: Digitization

As a starting point for the digitization the mannequin can vary in sizedepending on the major demographic of the customers for the retailer.Some retailers are more focused on larger men and women and then itmakes more sense to start with a larger mannequin and a larger sizedgarment. The closer to the starting point a visualized garment is thebetter the garment will look on the body model.

The “canonical” body model is a computer model representing the samesize as the mannequin used during the photo shot. In the ideal situationclothing items could be transferred directly from the mannequin to thecomputer model without morphing.

Section 64: Garment Positioning

For the garment to be represented in an accurate way on all of the eightviews and for the garment to be on the same horizontal level and notmove up and down in the views, key points on the body are marked. Thisis to have for instance a straight waistline across all of the views.

Section 65: Change the Model Size and See the Garment Fit

The body model can change body shape in a variety of ways, and duringthe post processing stage of the digitization the operator can changethe body size to vary to see how the digitized garment would look ondifferent body shapes.

The variations are offset from starting points on a 3D mesh and thosedifferent vertices have different modes of variation. Different ways toshow the variation can be to change the overall fatness, height, thighlength, breast size, hip width, and other modes of variation. Thevariations can be adjusted linearly and in combination. Each value canbe adjusted independently of the rest. This enables the possibility toachieve any body form and size within the limitations set by maximal andminimal values.

Section 66: Body Model

The body model (also referred to as a ‘virtual body model’) needs to beconstructed to be able to reflect the great variety of shapes of bodiesand also to be able to be changed and reconfigured between them.Defining the input data for the model is one way of defining what thebody model will look like and how to approach the model creation.

Another approach is to let the user upload a photograph where she isstanding in a doorway. Since the doorway is rectangular with straightedges, defining the doorway in the photograph allows for correction ofthe aspect in the photo. Knowing one measurement in centimetres, such asheight, allows for converting that into pixels in the picture. Knowingthe size in pixels in the photograph allows for taking othermeasurements from the photograph.

Another way to get input data for creating the body model is to use a 3Dsensor such as the Microsoft Kinect sensor to the Xbox gaming platform.This is one of many available depth-scanning sensors that can be used toobtain measurements. The sensor creates a depth map that can show anobject and the depth and size of that object. In the depth map everypixel has a value corresponding to the distance to the point in thescene from the camera.

See also Section 6.

Section 67: How to Turn Weight and Height or Other Input Data into aShape for the Body Model

The body model is in one example built up by facets, so it has verticesand triangles. The body model has 18,000 vertices and one of the stepsis to map the input data to those vertices to define the user's shape.

The model to map the input values is based on a probabilistic model ofhow the human body shape varies amongst the population. Some of thevalues are predefined such as that the model should have two arms andtwo legs.

This is done by starting with a mean model of the population which isthen adjusted with the population variation. The technique used for thisis principal component analysis (PCA). It can be understood that thismodel is used to find the most significant shape variation relative tothe mean.

The modes of variation can be the proportion of the variation over thepopulation. The most variation of the body shape is in height and thesecond most common is the body-mass-index (BMI) of a person: how muchbody fat each person carries. Other modes of variation in the model canbe torso length compared to leg length, the shape of the torso (appleshape, pear shape etc.). This can be considered as a low dimensionalmodel of body shape variation.

Using the dataset of body variations, principal component analysis isperformed on this dataset in order to use the mean body shape and themodes of variation in relation to that mean.

One database that can be used for the body shape modelling is theanthropometric CAESAR database of body shapes obtained through laserscans. This dataset provides the basis for the low dimensional model ofbody shapes. The low dimensional model can have for instance 10-20parameters. These are used to describe the modes of variation inrelation to the mean model. If the first parameter is overall scale, thesecond might be the BMI of the body. Having the 10-20 parameters willprovide good enough accuracy to generate a body model to a good standardof representation.

When fewer measurements are obtained for a body, the more approximatethe model will be. Using only height and weight and applying a model formaximum likelihood estimate of the body shape based on the given theheight and weight is possible. Since many body shapes can correspond tothe same height and weight, the model will present the most likely bodyshape for those two measurements.

Since the probabilistic model can provide several different probablebody shape models, one example is to present the user with a pluralityof body shapes based on the most likely body shapes and ask the user toselect the one that is closest to the body shape of the user. Thissample of plausible body shapes exemplifies a range of different bodytypes.

Another way of improving the body shape of the body model is to give theuser the alternative to select a more muscular or a fatter body shapeand use that for the calculation. This will provide input to the modelfor instance because fatter people carry more weight typically aroundthe tummy, thighs and butt, whereas a more athletic body shape carriesmore weight around the shoulders and the arms.

To further improve the body modelling more values can be inputted in tothe model. One way for this is to ask the user to provide moremeasurements, such as the waist size. The more values provided the moreconstrained the model becomes.

See also Section 7 and 8.

Section 68: Garment Size and Stretch

Different garments have different properties when it comes to size andstretch. This information can be used for modelling a particular size ofa garment to a particular body model and to give a realisticrepresentation of what the garment looks like.

For any garment that is smaller than the body model, information aboutthe stretch of the garment is needed in order to visualize the fit. Anygarment also has different stretch capabilities on different parts ofthe garment. For instance a hem will stretch differently from thestretch of the plain fabric of the garment. A garment can also havedifferent types of fabrics that stretch differently. Just knowing thestretch properties of the fabric of the garment is therefore not enoughto model the whole garment.

This problem can be overcome by taking stretch measurements of thegarment at different points on the garment. Those points can correspondwith the points where the user is asked to input the body sizemeasurement.

See also Section 127.

Section 69: Garment Sizing Table

It is understood that each garment will have an individual sizing tablefor the absolute size values for each garment. This table can further bepopulated with specific measurements, for instance hip, waist and chestand be populated for the different garment sizes. This enables one tomap the size closely to the body model size for a specific fit.

Different garments have different properties for stretch and also areintended to fit differently. A t-shirt may for instance be intended tobe worn loose or be intended to be worn more tightly fitted; a lycra topis only intended to be worn fitted and also has more tolerance forstretch. Some other garments are intended to be worn fitted but they donot stretch. So to map the body shape of the user tightly to anappropriate garment is important to be able to provide a good fit forthe garment.

Using garment specific fitting tables and by including stretch valuesand knowing the tolerances for the fit in the specific garment isimportant to be able to provide a good representation and also to beable to provide a good fit for the garment. This fitting table can bebuilt up with specific mechanical measurement equipment that measuresthe strain vs. stress for different parts of the garment.

This measurement is exemplified in FIG. 42, which shows that when thebody parameter is inside the garment parameter there is no stretch ofthe garment and it would not feel tight to wear. This is also shown forpoint A in FIG. 41A. At point B in FIG. 41a the body parameter is thesame size as the garment and the garment starts to feel tight to wear.

The stretch of a garment can also reach a point where increasing theforce will break the garment. Stretching is exemplified by path C inFIG. 41A. Also in FIG. 41A for path D it is shown that when reducing theforce not all garments will behave in the same way as when increasingthe force and will not return to the original shape in the same way.i.e. the stretching and shrinking processes do not follow the same pathsin the stress-strain diagram.

The sizing table built from the actual stretch properties of the garmentcan be used to give recommendations to what is the best size of thegarment for a particular user. It is also understood that thisrecommendation can be augmented with human input to provide arecommendation that also takes into account the subjective element ofwhat a good fit is. The subjective input can for instance be that aparticular garment has properties such that a particular user would wantto wear it fitting precisely. Knowing this will put the garment on thecurve in FIG. 41A at point B.

In one example the user can decide if they for instance would like thet-shirt to hang loose, closer to point A on FIG. 41A, or have a moresnug fit closer to point B.

Section 70: Providing Retailers with Specific Size Recommendation Tables

The virtual fitting room is able to provide data to retailers and brandson the body sizes of their specific customer demographic. The modes ofvariations of the consumers' bodies can vary between different retailersand this information can be gathered and provided. As an example forfast fashion brands it is more likely that the main mode of variation isthe height whereas for a brand for middle-aged people the main mode ofvariation is the BMI of the customers.

Section 71: Adapting Garment Modelling to Match with the User's BodyModel

Each user has an individual body shape and the process of adjusting thegarment is therefore central to the user experience. Each garment thathas been photographed has a model that is then annotated and can beadjusted to match with changes in the size of the body model.

When the body model deforms relative to the starting point, thesilhouette changes. This changed silhouette gives a starting point andan indication of how the garment model is to be modelled in relation tothe changed body model.

In an example, the garment information is captured in 2D and thegarments are also modelled in 2D.

Starting with the canonical body shape as exemplified in FIG. 43A, thebody shape of the body model has one specific characteristic shape. Thealternative body shape as exemplified in FIG. 43B has a differentcharacteristic shape.

A garment as photographed on a mannequin with the same shape and size asthe canonical body model will have a specific shape. To model thatgarment on an alternative body shape the garment needs to be transformedto illustrate the fit on that body model.

Seeing that the body silhouette is close to the silhouette of thegarment when modelled on to the canonical body model, it can be inferredthat the garment is tighter on those parts where the silhouettes areclose. Also the stress/strain properties that have been measured for thegarment enables the understanding that the garment is tight on the partswhere the silhouettes are close. It can also be inferred that thegarment is loose if the perimeter of the body is less than the perimeterof the garment.

Dividing the garment model in horizontal segments as shown in FIG. 43A,for each segment there is put a value on the fit of that segment. Whenmodelling the garment on to a different body model each of thosesegments can be individually modelled for the fit of that segment alongthe stress/strain curve as shown in FIG. 41A. The image of the garmentis then stretched locally for that section of the garment.

For example, section A in FIG. 43 is under tensile stress in FIG. 43Band the garment has stretched. When that section is modelled on to FIG.43B that segment of the garment model is expanded simulating that thegarment is stretched. This is exemplified by movement along the line atP in FIG. 41B.

In another example, section B in FIG. 43 is not under stress in FIG. 43Aand the garment hangs loose. This can be shown with a position along theline at Q in FIG. 41B. When that section is modelled in FIG. 43B, thebody will fill the empty space, then the garment is in the elasticregion, and the garment needs to be stretched. The garment model is thenstretched from the point where the body model meets the garmentperimeter. This can be shown with movement along the line at P in FIG.41B.

The sections of the garment model are also coupled together so thatstretch in one of the sections will have an effect on the neighbouringsections of the garment, as shown at R in FIG. 43. The model can furtherbe thought of as a series of springs and rods, where the springs do nothave a negative pressure i.e. they are not under compression.

The stress/strain measurements measure the inner geometry of the garmentwhich enables modelling of multi layer garments as well.

Section 72: Shadows

One aspect is that there can be a shadow on the thighs of the body modelwhen the model has a skirt on. The same can be applied if the body modelwears a long dress and the shadow is then laid onto the legs furtherdown. It is important that the shadow is consistent with the lighting ofthe garment and the artificial lighting in the body model. For instanceif the lighting is diffuse, a shadow might not be appropriate.

Section 73: Photography Process and Tool Example

User Requirements

For the photography process the minimum requirements are that theGarment Photography tool is installed and that the computer has minimumtwo working USB ports. An example of setting up the photographyequipment is shown in FIG. 28.

The steps to go through are described below:

1. Prepare Garment

Remove visible labels and stickers, apart from sewn in labels

Hang the garment on a hanger and steam out all creases

Check for loose threads, stains or other blemishes on the garment

Ensure that you know how the garment is meant to be worn (eg where thestraps and ties go) and if in doubt, photograph in more than one way

2. Dress mannequin

Choose stub arms to fill the sleeves as fully as possible, but ensurethat the stub arms do not stick out from the bottom of the sleeve andobscure the garment

Do up all zips and buttons unless you want a specific alternative style

Pull the garment down fully to ensure fit and to reduce folds in thefabric

Make sure that the garment is straight and looks as good as possible

A long sleeved Garments Exception example is shown in FIG. 29.

Long Sleeved Garments

A garment with a sleeve that is longer than the armpit is considered tobe long sleeved

This will need to be photographed twice, once with the sleeves down, andonce with the sleeves up

Try to keep the rest of the garment unchanged when you remove the arms

Make the folded up arm as flat as possible so that it does not obscurethe garment

Make sure that the folded up arm does not cover the armpit of the dressor below

2.2 A Translucency Exception is shown in FIG. 30.

All translucent garments must be photographed on both:

a) A white mannequin with a white background—as usual

b) A black mannequin with a black background—eg. use the black side of apoly-board to create the black background and use opaque black tightsand a long sleeved black top to create a black mannequin.

2.3 A photography process for shoes is shown in FIG. 31.

Shoes are not photographed on the mannequin

Shoes are photographed separately from each other so that they do notobscure the other shoe in some views

Place the shoe on a white covered box over the spikes on the turntableso that it is facing forward completely straight

Shoes must be photographed with a shape as close as possible to how theylook when worn by a foot—stuff the shoes with tissue paper/card but donot obscure any outside shoe:

3. Prepare Equipment

Check that the equipment is set up according to the specifications

Turn on the power sockets for all relevant equipment

Plug the camera and turn table cables into your computer's USB ports

4. Prepare Spread sheet, such as shown for example in FIG. 32.

This is to record garment information and photograph names

Ensure that the names that you give garments as you save them aresimple, logical and consistent

Ensure that the names in the spread sheet are exactly correct

5. Open the associated computer programme.

6. Capture Photographs.

The turntable will now rotate 360 degrees whilst eight photographs arecaptured at different angles.

7. Quality Control the Photographs as shown for example in FIG. 33.

8. Save Photographs.

10. Post Process Garment

The garment is now ready for post processing with the Garment ModelEditor.

Post Processing of Garments

For each of the eight photographs of the garment the sprites are cutout, the edges are cut out to make the garment free from the backgroundand the mannequin in the photograph.

See also Section 59.

Section 74: Mannequin Adaptation BACKGROUND

One other previous solution to this problem has been to wrap the armaround the mannequin's neck for instance. This has the problem that thesleeve still covers some of the torso material and there might also besome misalignment where the two layers in the photograph meet.

The meeting point of the two layers needs to be as much in alignment aspossible for the garments to display nicely. One approach could also beto tuck the leg in to the mannequins to be able to photograph thegarments.

Metail seeks to supply its users with accurate visualization and sizerecommendation of garments, through interaction with a software package,on-line or otherwise. Metail achieves this goal in three steps: firstlya three-dimensional model of the user is generated; secondly garmentsare photographed, processed and fitted to a default body (a step calleddigitisation); and lastly the digital representation of the garment isdeformed to agree with the users body shape gained from the first step.This description is concerned with a refinement to the second step,garment digitisation.

As mentioned above, garment digitisation can be broken down into threesteps: photography, processing and fitting. However they are linked inthat the quality of the previous step directly impinges on the output ofthe next step. For example if photography is performed to a low standardthe output of the processing step will also be of poor quality.Therefore taking high quality images is an important first step. In thesimplest case one image is required per “view” of the garment however aspecial case is garments which have more than one “layer”. By layer wemean a portion of the garment which covers, or is covered by, anotherportion of the garment. An example of this is long-sleeved garments,which have a sleeve closest to the camera. This hides a portion of thegarment torso, which in turn hides a portion (or all) of the sleevefurthest from the camera.

Long-sleeved garments need two images to be taken per “view”; this is tocapture all of the garment fabric. When the arms are down by the side ofthe garment, in six of the eight views, part of the garment torso isoccluded by the garment sleeves. To overcome this problem Metail hasdevised the following scheme: an image is taken of the garment with thearms up (secured away from the garment torso), from this a layer thatrepresents the garment torso can be taken. A second image is then taken,with the garment sleeves down next to the torso; from this image, twolayers representing the garment arms are produced. These three layersare then placed one on top of the other (the order depends on theviewing direction), and together forms the digitization of the garment.

The purpose of producing these different layers, and hence preserving adigital representation of the garment fabric in occluded regions, is toallow fitting of the garment to different body models. When the garmentis stretched or contracted by different amounts in different regions toagree with a user's body model, the different layers will stretch orcontract different amounts and so slide past each other. If a depictionof the occluded garment texture wasn't preserved in newly non-occludedregions then this sliding would uncover regions of the body with nofabric covering them. This would be an undesirable result.

The two images of FIG. 21A-FIG. 21B illustrate the way that twodifferent layers, namely sleeve and torso, can be stretched or shrunk inorder to match the body shape they are “worn” on.

Above, the example of a long-sleeved garment was used. However the samecould be true of any multi-layered garment, such as trousers. Thediscussion below uses long-sleeved garments as an example to clarify thedetail, however the skilled person will understand that each argumentapplies to other multi-layered garments.

Problem

Above, the details of digitising a multi-layered garment and inparticular a long sleeved garment were outlined. In this section theassociated problems with this scheme are expanded upon and in the nextsection a solution to these problems is detailed.

When obtaining the two separate images for a multi-layered garment,there is an important issue to overcome. This is the marriage of thelayers where they join, which for a long-sleeved garment is at theshoulder. To obtain a realistic join of the arm layer with the torsolayer the digital representation of these two layers must be identicalin the region of the shoulder. This is often hard to achieve as when agarment has its sleeve rolled or pinned up there is distortion of thefabric in the region of the shoulder. Alternatively the garment sleevecould be removed. However this has two disadvantages: the garment is noweffectively worthless and secondly if any of the sleeve-down images needto be reshot, this is now not possible. However a scheme has beendevised to allow the garment torso to be photographed while creatingminimal distortion in the region where the layers join, while at thesame time keeping the garment intact.

Solution

The solution to the problem outlined above is to create apertures in themannequin (either in the arm, torso or leg) into which occluding layerscan be placed when obtaining images of occluded layers. To illustratethe point: a long-sleeved garment might have its sleeve inverted throughthe arm scye (which is an armhole (or, occasionally, a leghole) intailoring and dressmaking) and tucked into the aperture in themannequin's side or arm. This, coupled with an attachment to themannequin shoulder, allows the garment to be identical at the shoulderregion in both arms up and arms down images. The aperture in themannequin structure is necessary as to allow the sleeve to be invertedwithout distorting the side of the garment torso, as would be the caseif it were simply inverted and allowed to hang or even tied to themannequin torso. In general, the occluding layer must be able to beinverted inside of the occluded layer and stored within the mannequinstructure so as not to distort the shape or appearance of the occludedlayer.

This scheme therefore produces the following results: the sleeve downimage and sleeve up image may be close to identical in the joiningregion of the shoulder, thus reducing image processing time andincreasing its success; the time taken to tuck the sleeve into theaperture is envisaged to be small, thus reducing processing time. FIG.22 shows a shoulder attachment and a possible location of an aperture.

Again it is emphasised that this scheme may be generalised to othermulti-layer garments. For example it is envisaged that an aperture mightbe created in the bottom of the mannequin torso and the legs made to beremovable. This is so that one leg, supporting the mannequin, may beshot, whilst the fabric of the other is placed in the mannequin'saperture. The roles of the legs might then be reversed for the secondset of images.

The advantage derived from this approach is to reduce processing time ofmulti-layered garments: this gives a competitive advantage as it willdrive down digitisation costs.

Depth from stereo to aid garment segmentation: Using depth from stereoimage capture (one of many means) as an aid to garment segmentation inthe 3D domain would require far less user intervention inpost-processing, even though we'll probably still want to represent ourmodels in the same way.

Section 75: Garment Segmentation and Alternative Methods

Garment segmentation is cutting out the garment from the background. Ifyou have stereo images you have more pixel information by which toautomatically understand what pixels belong to which part of the scene,i.e. background, foreground and garment. Our approach allows forfeatures such as taking image photography of the garments on real movingpeople rather than having to restrict the scene with a definedmannequin. This could help with retailer logistics. This also allows usto build a visual hull in 3D of a garment to give more information abouthow the garment then will move and react and deform depending on thesize of the person inside.

Section 76: Garment Translucency Problem

When digitizing garments with translucent or transparent sections it isnecessary to provide a method to deal with these sections.

Metail's present method of post-processing garments is to sectiongarments from the background (defined as anything which is not garment,i.e. mannequin, back-drop, lighting apparatus etc.) by means of a splinedrawn round the desired sections of an image. The image has a .jpg fileformat that consists of three colour channels (red, green and blue orRGB).

The problem of using this method of separating a garment withtranslucent sections from its background is that elements of thebackground will be included in the foreground. For example a blacktranslucent garment photographed on a white background will have pixelswith elements of white and black in them, giving a RGB value at somepoint on the greyscale. This is undesirable as the translucent sectionsshould respond to their background. For example if they are placed overa skin colour they should appear not grey but a mixture of black andskin colour.

Present Solutions

Two strategies have been formulated to overcome this problem. Both relyon changing the image format from three channels to four. Including analpha channel, that describes pixel opacity, allowing background colourto be expressed when the garment is laid over it. This so called ‘alphamatte’ contains values between 0 and 1, where 0 indicates totaltransparency and 1 total opacity. For pixels with an alpha value of 0,none of the foreground colour is represented and 100% of the overlaidcolour is seen. For pixels with an alpha value of 1, 100% of theforeground colour is represented and none of the overlaid colour isseen. For intermediate values of alpha a mixture of foreground andoverlaid colour is represented. The two strategies differ in the way thealpha channel, and therefore the alpha matte, is calculated.

The first method is for “whole-panel opacity”.

This is applicable when large areas of the garment (or perhaps the wholegarment) share similar opacity. In this case the garment is photographedon a background of a similar colour. At present this is achieved byfirst dressing the mannequin in tight fitting clothing of a similarcolour and placing a board of similar colour behind. For example a blackgarment dressed on a black mannequin with a black board placed behind.In the future it is imagined that a mannequin and background, whosesurface colour can be changed, might be used. The garment is then“splined” in the same way as a normal garment before its opacity ismodified manually by the annotator. The product is an alpha matte wherethe alpha value of each pixel outside the spline is set to 0 and thealpha value of each pixel inside the spline is the same and set by theannotator.

The second method is “per-pixel opacity”.

At present this method is applicable for loose garments of a singlecolour. The garment is photographed against a background of acontrasting colour, for example a black garment on a white mannequinwith a white background. In the future it is imagined that a mannequinand background, whose surface colour can be changed, might be used. Theimage is then altered by software. This software performs the followingsteps:

a. Estimates RGB values for the background and foreground.

b. Uses these values to estimate alpha channel values on apixel-by-pixel basis.

c. Subtracts a corresponding amount of background colour from eachpixel.

d. Normalises the colour in each pixel.

The product is an alpha matte which has alpha values ranging from 0to 1. The image is then splined as above. This results in an alpha mattewhere each pixel outside the spline has an alpha value of 0 and eachpixel inside the spline has a customised alpha value ranging from 0 to1.

Further Solutions

One further innovation has already been mentioned: using a mannequin andbackground of changing colour in image capture. It is imagined that thesurface of the mannequin might emit light at frequencies that can becontrolled by the operator. Alternatively the mannequin and backgroundsurface might reflect light of different frequencies in differentmanners, for example they might reflect light in the ultra violet regionof the spectrum differently to the way it might reflect light in thevisible region of the spectrum.

Another innovation is to use a more general method for estimatingforeground and background colours. For example, machine learningtechniques can be used to “teach” the software which RGB colour valuescorrespond to background, which correspond to foreground and whichcontain elements of each. In such a way it is understood that alphamattes for translucent garments of more than one colour could beproduced.

Another innovation is to use the ability to change mannequin colourwithout re-dressing it (the first innovation above) to create alphamattes in the following way. Two images of each garment in the sameposition will be taken. The first image will have a background of acertain RGB value, the second image will have a background of adiffering RGB value. The difference in RGB value between the two imageswill be calculated on a per-pixel basis. Those pixels that have adifference known to be the same as background pixels will be labelledbackground and their alpha value set to 0; those pixels that have adifference of 0 between the two images will be labelled foreground andtheir alpha value set to 1; those pixels which have an intermediatedifference will be labelled as translucent and their alpha value setbetween 0 and 1 corresponding to the magnitude of the difference betweentheir RGB values. In such a way an alpha matte of the image will beproduced.

It is believed that this last innovation will provide the most generalsolution for creating alpha mattes of translucent garments.

Other Features Include:

Locating the subject silhouette automatically

The photographed subject, for instance the garment on the mannequin, hasa silhouette that can be located automatically using an edge findingapproach.

Using detents to assist in mannequin positioning

Any mechanical means of making mannequins easier to assemble or clotheseasier to photograph, e.g. the idea of using detents for getting armposition right even after the mannequin is dressed. This is importantfor having an exact position of the mannequin between differentphoto-shoots.

The use of camera calibration technologies with a reference frame

The use of (various) camera calibration technologies to facilitateprecise alignment of image-based garment models with a reference frame.

The use of green screen and/or colour segmentation technology tofacilitate garment digitization for visualization applications.

The use of image morphing to facilitate the adaptation of image-basedgarment models to fit real human bodies. Fitting clothing to body modelswith respect to size and a visual illustration of how the item would fitthe individual.

The use of a low dimensional representation of human body shape tofacilitate capture of a user's body shape via a simple user interface(UI).

The information gathered from users uploading their photos can be usedto create body shapes via a simple UI. The system possesses informationabout common body correlations allowing for more precise body modelseven if the information is insufficient.

Section 77: How Metail Uses Metadata for Hairstyles

Hairstyles have attributes set to configure how they are coupled withthe virtual fitting room Generic Modeling Environment (GME) “garment”ids, and how they are filtered in the UI based on skin tone andhairstyle.

Examples of metadata files are shown in FIGS. 34 and 35.

Metadata needs only to be edited for one view (eg. view 0) of any onemodel— it will be applied to all subsequent views automatically. So alsoassociated with the hairstyle in FIG. 34 are views 0 to 7.

Section 78: How the Virtual Fitting Room Use Metadata for Hairstyles

The virtual fitting room use hairstyle metadata (text strings) topopulate information against hairstyles for the following purposes:

-   -   1. To link hairstyles in the Vizago system to morphing data in        the virtual fitting room system (vs_id) and set important        variables such as viewpoint so that the hair can be applied to a        face and the head attached to the body in the right place.    -   2. To group hairstyles using the name (for the views associated,        so that the same hairstyle but correct data is applied to each        view of the 3d face).    -   3. For filters (both shown and hidden in the user interface) the        following are used— screens. Where they are used is shown below        -   a. Hidden filters: Some hairstyles will only work with pale            skin faces, some with dark skin. Setting the metadata for            each restricts the selection of available hairstyles so that            a user can choose only compatible hairstyles with their            complexion (compatible in this context translating as those            which give the best results). The relationship is one to            many (one hairstyle to multiple skin types where relevant)        -   b. Shown filters: The user can drill down when selecting a            hairstyle through setting categories to filter the hair on.            The xml data is used to set which categories the hairstyle            should belong to. The relationship is one to many.

Section 79: Key Features of Social Media Integration

There is provided a method of enabling users to interact with virtualbody models, comprising the steps of (a) providing a virtual dressingroom in which a first user can select one or more garments to visualiseas being combined with their virtual body model; (b) making thatcombined virtual body model and garment available to other users thatmeet defined collaboration, sharing or friendship criteria set by thefirst user.

The method may be one in which a first user's virtual body model isshown or shared with another user so that other user can select and buyclothes to fit the first user, whilst concealing actual measurementsfrom that other user.

There is also provided a method for sharing a virtual body model of aperson, in which the virtual body model is generated by the personinteracting with a web site, and in which the virtual body model isaccessible or useable by a different web site, namely a socialnetworking web site.

The method may be one in which the person has selected a garment on agarment retail web site and that selected garment is visualised on thevirtual body model, and in which the garment retail web site includes abutton or icon which, when selected, automatically posts an image of thegarment visualised on the virtual body model to the social networkingweb site page, entry or resource of that person.

The method may be one in which the social networking web site page,entry or resource of that person enabling various functions, such as (i)showing what the user is wearing or thinking of wearing and/or buying,allowing friends to comment, express a like; (ii) enabling friends todress the virtual body model with their own suggested clothing; (iii)enabling friends to add their own clothed virtual body model to theuser's virtual body model so that the user's page shows both modelstogether in their chosen garments.

The method may be one in which a first user's virtual body model isshown or shared with another user so that other user can select and buyclothes to fit the first user, whilst concealing actual measurementsfrom that other user.

There is also provided a method for sharing a virtual body model of aperson, in which the virtual body model is generated by a userinteracting with an application or web site, and in which the virtualbody model is accessible or useable by any type of display device thatis operable to use an extensible image framework, such that images of agarment can on any such display device be seen as combined onto thevirtual body model, to enable that person to visualise what the garmentwill look like when worn by them.

The method may be one in which the extensible image framework includes aweb browser. The method may be one in which the extensible imageframework includes a browser API or extension. The method may be one inwhich the type of display device includes one or more of: televisions,mobile telephones, tablet computers, laptop computers, desktopcomputers, digital mirrors, digital cameras, digital video cameras,displays placed in retail stores, digital mirrors placed in retailstores. The method may be one in which the type of display deviceincludes a television and a tablet computer. The method may be one inwhich, when the user is viewing an image of their virtual body modelcombined with one or more garments on a portable device, the user can,with a single physical gesture, cause the same image to be displayed ona different display device. The method may be one in which the gestureis a flick or throw gesture.

Section 80: Social Interaction

The virtual fitting room can in different examples have a socialinteraction aspect. One example is shown in FIG. 10-11. The concept ofsharing is part of the interaction on the Internet and there are toolsto share content with others.

Section 81: Social Needs

The social interaction is an important aspect of the platform and thetechnology. There are different reasons for different users as to whythey interact socially with the service and the platform. Among thedifferent social drivers for interaction four are more prominent: 1)purely social, 2) exhibition, 3) recommendation, 4) discovery.

‘Feedback’ is the idea that you get in-depth feedback from your widersocial circles.

‘Discovery’ facilitates you to find out new fashion items. ‘Try thatlook on me’ is a discovery tool.

‘Exhibition’: the idea to show off how wonderful your fashion taste is.Try to publish your looks to as many people as possible.

‘Recommendation’: you view yourself as an expert, as a fashion stylistor expert. Use the virtual fitting room to be able to giverecommendations to other users.

Section 82: Purely Social

Humans are social beings and interaction with other people is animportant part of human life. Many of the actions and choices we do arepurely based on interaction with other people. One aspect of the currenttechnology is to be able to interact and play and exchange experienceswith other people. The main driver for the user would then be just thesocial interaction in itself.

See also Section 93-96.

Section 83: Exhibition

A different type of driver for some users is that they would like toexpose themselves and get attention. The users could aspire for theattention through for instance being provocative, being true or in someother way get attention in some form.

Users of the system described in this document can in one example sharetheir body model as it is or with a specific outfit. This could be inthe form of a picture of the outfit as exemplified in FIG. 27 or itcould be in an interactive form where the other users could rate andmake comments and in one example change the outfit and features of theshared body model.

See also Section 89.

Section 84: Recommendations

Using information from other users, possibly from users you trust andbelieve you share common views with, is one way of social interactiononline. This is especially important online since the amount ofinformation is vast and you sometimes would like either to share yourown views to someone or you would like to get information from someonewho you trust or share views with.

Recommendation can also be as part of a crowd-sourced effort whereseveral users put in recommendations around a question, a subject orsimilar and the common view would then be the recommended view.

Using recommendation as the social driver can be implemented in that theuser either can recommended for example outfits for another user's bodymodel or the user can let other users recommend for example outfits forhis or her body model.

See also Section 30 and 98.

Section 85: Discovery

As stated above the amount of data in the online environment can be vastand it is sometimes hard to pierce through the “noise” to find what isreally relevant or to find new experiences. Discovery is to experiencesomething new which you might not have known already existed or at leastdid not actively contemplate.

In a solution, this could be in the form of experiencing a new outfitfor your body model. This could be through looking through differenttypes of outfits or garments and trying them on or through for instanceusing a randomized or curated selection of garments or full outfits.

See also Section 99-101.

Section 86: Online Social Interaction

The social interaction online has taken the form of different ways torecommend, share and interact with other users. There are differentinteraction tools that users can use. For example different icons can beselected, representing for example the social networking site Facebook,the micro blogging platform Twitter and to share an experience viaemail. All three of these icons would enable the user to share thecurrent experience which could be a web page, a photo, a video or someother experience which can be shared with another user.

Different online tools are available for sharing content andexemplifying the different range of media types for sharing andplatforms where content can be shared.

One implementation of the sharing function can be seen in FIG. 39 whereicons for sharing the look are present in the lower right hand corner.

Section 87: User Profile

In the virtual fitting room community, the user would be able to see hisor her profile. In one example the profile page would contain the bodymodel currently associated with that account as well as the wardrobe andthe looks being saved to that account.

The user would also be able to see the comments and rating she hasreceived in an inbox and it could also be presented as a flow of events.One part of the profile can contain the stylists' recommendations thatyou subscribe to or that you have received. The profile page can bedivided in to three main parts: stuff that you have done yourself, stuffthat has been done by your friends (inbox) and stuff that stylists havedone. The profile page would also be able to show notifications of whathas happened since the user last logged in.

An example of the profile can be seen in FIG. 10 and an alternative wayof visualizing looks can be seen in FIG. 11.

Section 88: Social Interaction Features

The following social interaction features may be implemented by thesystem:

Save Looks and Later Select Looks to Share

Save multiple looks and publish a selection with titles, comments andquestions for other users to view or interact with. This relates to thesocial need feedback.

Save multiple looks to the user's account so the user can access thelooks and share the looks as and when the user prefers. The saved lookswill be displayed in an area of your account where you can browse aroundsaved looks.

The user can share the saved looks using the sharing tools the virtualfitting room offers.

The saved looks can have areas associated with them for the user toenter comments or viewing feedback the user has received on a specificlook when the looks previously has been shared. The user can in oneexample also view and follow links to when and where the user has sharedthe look.

The user is able to access, store and sort the advice she is being givenfrom other users and for instance style advisors, in the virtual fittingroom.

Save Other User's Look

The virtual fitting room also provides for saving looks that you havereceived from another user or that you have selected in a store. Thelooks can for instance be recommended looks from fashion stylists or theuser's friends.

Statistics on Shared Looks

The virtual fitting room can also provide statistics on the specificlooks such as how many, when and from where other users have interactedwith the look, such as having viewed it or rated it.

The virtual fitting room further provides for the user to be able tofollow what garments have been rated highest or viewed the most whenincluded in a look.

Vote on preferred look, when shared through Facebook, email, twitter etc

Allow for voting when a user shares multiple looks using buttons(optional when sharing). This relates to the social need Feedback.

One example of the user sharing a look and allowing other users to voteis shown in FIG. 8-9.

This can be in relation to a specific event where multiple shared looksfrom the event are being presented and users can rate the differentlooks. This could for instance be from the Academy Award event or awedding where the dresses used by the people at the event can bepresented for users to comment on and rate.

The virtual fitting room also provides for the user to try the look onto the body model. If the dress would not fit the user could be notifiedthat the dress is not in the right size.

Allow Specific Questions to be Asked when Seeking Feedback

The user can pose a specific question or attach an issue to the look tobe shared. It can for instance be a question about “Which outfit wouldbe best for . . . ?” and the user can allow friends to vote for if thisis a good look for that purpose. This relates to the social needFeedback.

Notifications for when Feedback is Received

In order for the user to be aware that she has received feedback on alook an email can be sent out to the user's preferred address with anotification. The email can contain an image of the look with thefeedback. The email can alternatively provide an instruction to log into the virtual fitting room to view the feedback provided. This relatesto the social need Feedback.

Feedback Inbox

The feedback inbox stores all previous looks and feedback to thoselooks. The virtual fitting room provides for the feedback on looks to beviewed in an inbox type style where the feedback can be shown accordingto the chronology in which they were received or for instance the typeof feedback it is (rating, comment etc.). This relates to the socialneed Feedback.

As previously described, the feedback and ratings etc. can also beviewed in relation to each look or garment which the feedback relatesto.

Section 89: Sharing the Private Wardrobe

The virtual fitting room allows for the user to create a privatewardrobe to store garments and to share that wardrobe. The people thatthe user selects to share the wardrobe with can be friends or followersof that user.

In another example, the user can select to share the wardrobe with otherusers who have a similar body size to the user. The user can also selectto share her looks or specific looks to other users based on the bodyshape of those other users.

The filtering of whom the user will share the wardrobe with can be setby the user. This relates to the social need Feedback.

Adjustable privacy settings for sharing body for gift shopping: Thisrelates to the social need Feedback.

In one example, the virtual fitting room enables the users to share thebody model, not the actual measurements, with someone else. That otherperson can then shop for the user and get the correct size withoutknowing the exact measurements of the user. The user can set the levelof detail of the body model the user would like to share with the otherperson.

The user can also select to only share a standard image or a dressedimage of the body model to the other person. The measurements wouldstill be correct for the fitting of the garments the other person triesout on the body model but the other person is not seeing the actual sizeof the user.

In another example the user can share with the other person only thesize of the body model but not the actual body model. The other personis not able to change the body model's shape and size, unless the userallows this.

See also Section 83.

Section 90: Circle of Friends

Create a group of friends who can all chat and share collectively, butwithout wider publication.

This is a feature where the user can share looks with a closed group ofother users, friends, who then can make comments and rate the look. Thegroup can be closed for other persons, either totally closed or withdifferent levels of participation by other users. One setting can be toallow other users to see the shared looks but not interact with thelooks and make comments and rate etc.

Section 91: Get Live Feedback on Look from Friends

This relates to the social need Feedback. This feature is a real timechat with other users where they can see the user's look either as thelook is created or when the user elects to share the look. The livefeedback can for instance be in the “Facebook chat” format where activeusers can interact.

There can also be a function where the user can randomly select someoneto interact with, not necessarily from the set of friends. Thisinteraction can be in the form of a chat and the user can after thatrate the user having given the input on the look. This feedback can formpart of the overall rating of the user, such as points given for goodreviews or a helpful user.

Section 92: Fashion Panel AKA Style Counsel

Ability to create a group of friends to allow for one click sharing“share with my style counsel” This relates to the social need Feedback.

The fashion panel can consist of users that do not necessarily know eachother. The user sharing a look will push the look to the fashion panelfor them to make comments and provide feedback on the look. In oneexample the user sharing the look can allow the fashion panel to alsoalter the look and make new suggestions on how to improve it.

Section 93: Shared Live Fitting Room

Space where a defined circle of friends can see each other's bodymodels, recommend garments for each other to try on and instant chat.This relates to the social need Purely Social.

This can be seen as a shared shopping experience, an online shoppingtrip, for instance if you are shopping for a bridesmaid outfit togetherwith friends.

Section 94: Instant Chat Through Facebook (FB) Chat

Via the virtual fitting room the users can chat instantly with eachother. This can be used together with other collaboration via theplatform. The users can for instance chat with each other at the sametime as they are reviewing a specific look or going through a library oflooks or garments. This relates to the social need Purely Social.

Section 95: Invite Friends to Shop with Reminder

The users of the virtual fitting room can add events to a calendar andalso invite or include other users to an event. The user can also setthat the participants to an event will get a reminder that the event issoon to happen. An event can for instance be a joint shoppingexperience. This relates to the social need Purely Social.

Section 96: See which Facebook Friends are Using the Virtual FittingRoom

If the user has connected the virtual fitting room account to anothersocial network such as Facebook the user can see if her friends on thatother network are online. This relates to the social need Purely Social.

For Section 93-96 also see Section 84.

Section 97: Become a Stylist and be Rated by Peers

A user of the virtual fitting room can allow other users to follow theirprofile and get recommendations from that user on garments and outfits.Other users can also rate the recommendations given by a “stylist”. Thestylist rating can be displayed on the member's profile or in a list ofthe highest rated stylists. This relates to the social need Exhibition.

Section 98: Stylist Recommended According to Style Data

This relates to the social need Recommendation. Depending on the sizeand shape of the user's body model the user can be recommended differentstylists.

The stylists can in one example enter for what shapes and sizes theywill create recommendations. In another example the system derives forwhat shapes and sizes the recommendations are made, based on forinstance the size of the body model the stylist is using.

See also Section 84.

Section 99: Try that Look on Me

When looking at a stylist's look book/posting, there is a “try this lookon me” button. This relates to the social need Discovery.

People could also become “stylists” and share their recommendations tothe wider crowd. Users can also select to subscribe or follow a specificstylist and that way get new recommendations to try on the body model.

In relation to shared and public looks the user can in one example clickon a button or link to try that look on the user's body model.

Section 100: Follow Friends/Celeb Looks

As described a user can allow other users to follow her profile and theuser can give style advice. In one example the user needs to approve thepersons following her profile and her style advice.

Users can be rated and get recognition through being assigned differentlevels of credibility. One aspect of this is that the stylists canaspire to those higher levels through for instance user recommendationsand the number of style advice given. The stylists can be awardeddifferent credibility badges, such as stylist guru or a person givingreally niche style tips.

In one example there are different levels of for instance followers withgoals that trigger different real value recognition items such asvouchers at a specific retailer. The user could also be recognized forpushing a specific garment and that people buy that garment afterviewing it or after getting a style recommendation for that garment.

The virtual fitting room provides for the stylists to create their own“world” within the fitting room where different features can bepersonalized. The stylists and other users can also become affiliatesgetting a share of the sales generated by traffic they have driven.

Section 101: Like and Dislike Stylist Recommendations to Find BestStylist for You

The virtual fitting room enables the users to provide feedback about therecommendations the stylists have given. This information can be usedfor the public rating of the stylist and also for the service tounderstand the effectiveness of the recommendations made. This relatesto the social need Discovery.

For 99-101 also Section 85.

Section 102: Look Recommendations Connected to a Specific Event

The outfit randomizer can also be related to events such as films andother things that are happening in the media. It could be to dress up asif you were in a film or as if you were going to the Oscars night.

Section 103: Different Social Platforms to Share Via Facebook®

The system could have a specifically designed Facebook® welcome pagethat the user would land at when visiting Metail's fan or group page onthe social networking site. There could be different landing pages withdifferent features for different social networking sites or services.

The user has the option to also share the look on to a wide range ofplatforms. An example is sharing an outfit on Facebook. The user wouldclick on for instance the Facebook logo and the look is then presentedon the Metail website in relation to the look the user would like toshare. It is understood that the interface the user would be presentedwith will differ depending on the social networking site used also onthe current features and layout of the site. For example a text box isprovided where the user can write a message to be shown in relation tothe shared look on Facebook.

The look can be shared on the wall of the user to be viewed by a widerange of users or the look could be shared as a personal message to onespecific user or group of users.

The shared look can include some standard elements in accordance withthe Facebook requirements such as a title, a displayed universalresource locator (URL), a correct link to the shared look, and an imageof the shared look.

The thumbnail associated with the shared look on Facebook would be astatic image that either is preselected by the system or the user canflick through alternative thumbnails to select it for use.

The shared URL will take the user to a webpage where the look is shown,as exemplified in FIG. 27. The information on the shared look page canbe customized in many different ways. The webpage in FIG. 27 has a linkto the virtual fitting room start page for any user to create their ownbody model. The page also has a “like button” which will initiate thelike function on Facebook.

In one example, information about the garments in the shared look isalso accessible from the shared look. This can be in the form of a listof the garments the body model is dressed in.

The information about the garments the body model is dressed in can beprice, retailer, brand and size. The user can in one example set whatfeatures should be shared together with the specific look. In anotherexample the sharing preferences are set centrally, e.g. by the brand orretailer or by the platform administrators. In another example therecould also be links to the retailer or to the garment in the virtualfitting room.

In one example the user can select to share several views of the samelook and all or a select subset of the views of the look would be sharedover the social networking site. The user can either share a subset ofthe views available or all of them. In one example this lets the personviewing the page with the shared look, eg. as shown in FIG. 27,alternate between different looks—rotate the body model in the outfit.In one example the background of the shared look is changeable by theuser.

Other Options for Sharing

In one example the user is able to choose between different views of thesame look for the image of the shared look. This can be done eitherthrough viewing the look the user would like to share on the Metailwebsite and activate sharing and that image would be the default to beshared. The user can alternatively in one example select between severaldifferent images to be shared and the user could then click through thedifferent images on the stage of sharing to select the image the userwould like to represent the shared look.

In one example the user can select to share several different looks overthe social networking site. The looks, and if selected also alternativeviews of the same, are shared as a set of images. This can be in theform of an album on the social networking site or as a linked album orstream of images on a different networking site.

Twitter

The user can select to share the look via a micro blogging platform suchas Twitter by selecting an icon. The user is then presented with theoption to share the look using the link to the page (eg. in FIG. 27) viathe Twitter interface. The user is presented with a standard form textreading “View my look from the Metail fitting room, and create yours”;alternative text can be used. Also in the standard format presented tothe user is a URL which leads to the shared look. It is understood thatthe URL can be shortened and it can also be presented as such: theactual landing page has a different address.

The user can alternatively select to share the image of the look onTwitter using for instance the tweet pic sharing platform or any otherplatform where the user can upload the image of the look. The tweet caninclude a specific identifier of the shared look; this could be a codeor a hyperlink.

Look Feed

In one example the user is able to share looks in the form of a feed.The user can in relation to a look as shown to the user in the Metailsystem select to publish the look to a feed. The look would then beshared on that feed: Other users who follow the user who has publishedthe look are able to see the shared look.

In another example the feed is in a form of a photo feed where the useruploads the look to an album and where the looks can be presented as afeed of image representations of looks.

Section 104: Email

When sharing for instance an outfit or a body model via email the useris presented with a template email to be sent. The template will in oneexample not allow the user to edit the content. In a different examplethe user can add specific text to the email in for instance a text box.

In yet another example the user can edit the email to be sent throughhis email client of choice and via Metail's sharing feature be presenteda link or a section of link and text to be pasted in to the email.

Section 105: Share Alternative Looks

In another example the user can select to share a plurality of differentlooks to appear so that other users compare the looks.

The multiple shared looks would in one example appear next to each otherand the page would provide for a comment and ratings option. The usersharing the looks could provide a text describing each of the looksand/or an intro text to the page. The user could then for instance posta question on Facebook saying “which of these two looks is better on me”and then link to the page with the two looks on the body model.

Section 106: Share the Outfit

Share the outfit as being presented on the body model and let othersdress their body model in that outfit.

The user sharing the outfit can select to share the outfit with aspecific friend for that person try the outfit on their body model. Theshared outfit can be presented on either of the user's body models. Theuser sharing the outfit can set whether she would like to share her bodymodel in the outfit or not. The user receiving the shared outfit doesnot have to already have a body model in the system, and if not she canbe asked to create one to see the outfit on a or her own body model. Theshared look could alternatively be presented on a standard body modelnot having the physical characters of the user sharing the look.

In one example the shared look would be presented as shown in FIG. 27with the addition of a button or link allowing for the user looking atthe shared look to try the look on to his or her body model, or tocreate one to see the body model dressed.

The background is part of the experience shared but can also bedecoupled from what is shared and what the user would allow other usersto use from her outfit if they decide to try the outfit on to their bodymodels.

Section 107: Sociability

The social interaction of the virtual fitting room is exemplified inFIGS. 10 and 11.

Some key features of the social interaction of the virtual fitting roomare listed below:

1. Share feature allows you to send your looks to Facebook and Twitterand receive comments and advice from friends whilst promoting thefeature

2. Combining fashion and beauty trends with personalised recommendationfor face shape, skin tone and hair type

3. Publish looks to the retailer's and/or the virtual fitting room'swebsites and receive comments and advice from the community

4. Community rating of beauty blog posts, tips and products

5. The virtual fitting room collaborations with stylists and fashionbloggers will add to the quality of advice provided by and to thecommunity

6. The community will grow as more brands are linked to the virtualfitting room, allowing for a sociable shopping experience online

Section 108: Share Your Model

The user can select to share with different circles, either createdwithin the platform or from other platforms. Decide different levels ofwhat you would like to share with an individual or list/circle offriends. This can be your outfit on your model, your outfit that someoneelse could try that on to themselves, the measurements, share the modelbut not be able to see in underwear.

The user can further select to share via an email list. This can forinstance be to share a look with friends who are not connected via asocial network or on the virtual fitting room. The user can allow aperson to share outfits to be dressed on to the other person's bodymodels. The user can also allow another user to dress her body model.The user can set the level at which another user can interact with thesharing user's body model. It can be to only change outfit, or to getthe actual measurements and size measurements. The other user couldperhaps shop for garments for the shared body model. Not showing thesize of the model during the process at all, all the way through thepurchase process. The user can share the shape and/or the measurements.

The user can further let a specific user access and dress the sharinguser's body model. This can be a mother dressing and shopping for achild and then interacting with the child's body model. Could also befor husband and wife.

Connect several model to one user account

The system allows in one example for a user to have more then one bodymodel associated with one user account.

Section 109: Key Features of the Virtual Body Model

There is provided a method for enabling garments to be visualized on avirtual body model of a user, in which (a) a display device shows animage of a garment, of a size selected by the user, combined onto thevirtual body model to enable the user to visualize what the garment willlook like when worn by them and (b) the display device also provides anicon, button, function, sliding scale or other control that, if selectedby the user, causes the image of the garment to be altered to correspondto what a garment of a different size would look like on the virtualbody model.

The method may be one in which the user can select one size larger andone size smaller. The method may be one in which the control is voiceactivated using a speech recognition system. The method may be one inwhich the image of the garment is generated by photographing in 2D anactual garment from a number of viewpoints, on a mannequin of known sizeand shape. The method may be one in which the garment is photographedfrom between 5 and 12 different viewpoints around the garment and theresulting photographs are analysed and processed to generate a 3Dphoto-real image of the garment that can be viewed from 360 degrees. Themethod may be one in which the image of a garment is a 3D photo-realimage of the garment. The method may be one in which, for a givengarment, only a single size of that garment is photographed and theappearance of other sizes is calculated by extrapolating from thatsingle size. The method may be one in which the process of extrapolatingis based on measuring other sizes of that garment, or differentgarments, from the same manufacturer of that garment.

There is provided a method of generating a virtual body model, in whicha user takes, or has taken for them, a photograph of their face which isthen processed by a computer system to generate a face image derivedfrom that photograph, in which the colour and texture of the skin of theface, obtained from or matched to the photograph, is then applied by thecomputer system to a virtual body model such that the colour and textureof the skin of the user's virtual body model is matched to the colourand texture of the skin of the face of the user.

The method may be one where the colour and/or texture of the skin of theface is obtained automatically from the photograph. The method may beone where the colour and/or texture of the skin of the face is obtainedmanually from the photograph by a user manually selecting anappropriate, matching colour from a palette of possible colours and/orpalette of possible textures. The method may be one in which the colourand/or texture of the skin applied to the virtual body model is inaddition a function of the age or the user.

There is provided a method of generating a virtual body model, in whicha user takes, or has taken for them, a photograph of their face which isthen processed by a computer system to generate a face image derivedfrom that photograph, in which the colour of the skin of the face and/orthe eyes, obtained from or matched to the photograph is analysed andused as a filter or criteria the system automatically applies whenrecommending a colour for a garment to be selected by the user from anonline garment retail website, or rating the colour of a garmental-ready selected by the user from the online garment retail website interms of complimenting the user's skin and/or eye colour.

Section 110: Aiding Body Shape Prediction Through Specific Features(Pregnant Etc.)

The model used to create a body model can be aided through addingspecific features about the user. This can be in the form that the userinputs that she has an athletic body or a fatter body. A similarapproach can also be used for modelling a pregnant body. Allowing theuser to input that she is pregnant will provide the body model withprobabilistic input about the shape of the body. An alternative versionis to model a pregnant body in a set of (for example weekly) stages tobe added to the body model to model fit and size for a pregnant woman.

Section 111: Improved Body Modelling Using Aggregated Customer Data

The aggregated customer data is used to improve body modelling bymapping information about generic similarities and correlations in bodyshapes. The information is derived from millions of users' 2Dphotographs rather than doing expensive collection of a few hundredbodies in 3D. The information is in one example used to generate themost likely body shape for user who hasn't supplied a complete set ofdata.

Section 112: Modelling Clothes that do not Fit

The virtual fitting room can in one example provide for the user to tryon garments of a size that is not the best fit. The system willgenerally select the garment size which would give the best fit to thesize of the user's body model.

However, providing the option for a user to try on different sizes willhelp the users in their decision on which garment has the best fit forthat specific user's body model. Some users will like the garment to bemore loose fitting than what others would find a good fit and thereforeto provide one size larger than the best fit to be modelled on the bodymodel would provide value to the user.

The virtual fitting room will in one example provide for the option forthe user to alternate size. This could for some garments be one size upand one size down. For some garments the range of available sizes to tryon the body model can be broader.

Section 113: Creating the Face

Annotation points will determine the extremities of the face and thedifferent points to be used in the process. The photo will be pasted onthe head of the hairstyle model which has been used. This is also whyonly certain hairstyles are allowed to be used with certain skin tones.

The transition between the face from the user's picture and thehairstyle model is done in the skin area and that is why the skin toneis important for the selection of the hairstyle model to be used. A toolightly coloured face to be put on to a dark skinned model will not givea good transition and vice versa.

Skin tone values are sampled from the face and from the model and amiddle value is given for the blend in the transition between the faceand the hairstyle model. The skin tone is sampled from a set of keyareas to give an accurate skin tone measure. In one example the sampleareas are the chin, the forehead and around the ears.

See also Section 4.

Section 114: Skin Textures and Use of Face Photo to Generate SkinColoration for a Body

Different skin textures can be applied on to the body model in order tomake the body model more realistic and to resemble the user.

The canonical body model can have skin textures such as shadows or skinfeatures such as the belly button and nipples applied to give a morenatural look. These features are in one example pre-applied and notchangeable by the user. In another example the user can select from aset of different body textures to be applied to the canonical model. Inyet another example the skin textures to be applied to the body model,either automatically or by user selection, are set by the selectedhairstyle and/or the user's skin tone.

Also, different body shapes will typically have different skin textures.For instance a muscular body will have skin texture features that aredifferent from an overweight person. The system can apply certain skintextures and/or features based on the body model's BMI, weight, lengthor other values.

In order to model a male body model naturally the option to apply chesthair and other body hair is provided for. The user can either choose toapply body hair for the whole body in different levels entered by anumber or for instance a slider. The option for the user to selectcertain areas to apply the body hair to can also be provided for. Theoption to remove hair in certain areas of the body may be provided for.

The colour of the face is an indicator of the colour of the rest of thebody. Taking the total skin tone value, an estimate of the skin colourfrom the face will give a value that can be used to colour correct theskin texture of the body model. This colour will also be used to adjustthe colour of the associated head onto which the face features aremodelled.

Section 115: Male Vs. Female Model

Building a simple probabilistic model of the body shape can beindifferent to if it is a male or female body model. Different inputvalues are needed for modelling male models instead of female, forinstance bust measurements for females. The bust size is one aspect ofthe body model that is likely to vary in a more random fashion and isnot necessarily related to the size of other body parts. Using inputvalues for the cup size and strap size provide values for the modelling.

Section 116: Key Features of the Application Programming Interface andMobile Aspects

There is provided a method for visualising a garment on a virtual bodymodel of a person, in which an icon, barcode or symbol associated with agarment can, when scanned by a mobile computing device, automaticallycauses one or more images of the garment to be combined onto a virtualbody model of the person to enable that person to visualise what thegarment will look like when worn by them.

The method may be one in which the icon, barcode or symbol is associatedwith a printed or electronic image of the garment. The method may be onein which the icon, barcode or symbol is in a printed magazine, such as afashion magazine. The method may be one in which icon, barcode or symbolis on a physical tag or label for the garment. The method may be one inwhich the icon, barcode or symbol uniquely codes for the garment. Themethod may be one where the image of the garment when combined with thevirtual body model is displayed on the same mobile computing device,which scanned the icon or symbol.

There is also provided a method of generating photo-realistic images ofa garment combined onto a virtual body model, in which (a) a userlocates a garment on a website; (b) a computer implemented systemanalyses the image of that garment from the website and then searchesand identifies that garment in a database of previously analysedgarments and then combines one or more virtual images of the garmentfrom its database onto a virtual body model of the user and thendisplays to the user that combined garment and virtual body model.

The method may be one in which the user, when viewing a website showingthe garment on a touch screen device, indicates that the garment is tobe visualised on his or her virtual body model by touching the image ofthe garment shown on the touch screen display.

The method may be one in which the computer implemented system searchesand identifies that garment in a database of previously analysedgarments by using an image matching system that can assess thesimilarity between two images of objects. The method may be one in whichthe computer implemented system searches and identifies that garment ina database of previously analysed garments by using garmentidentification metadata exposed by the website. The method may be one inwhich a web browser extension enables the user, when browsing thewebsite, to initiate the tasks required to be performed by the computerimplemented system. The method may be one in which the combined garmentand virtual body model is shown within the same web browser. The methodmay be one in which the combined garment is shown in a window appearingover the website.

Section 117: Mobile

The virtual fitting room is possible to view and interact with fromdifferent types of platforms, where the user will get the virtualfitting room experience from any device used. Using a multi channelapproach aids the different core features of the different devices andalso the different types of features that you would like to use when youare using a mobile phone for instance, in contrast to sitting at yourdesk by your computer.

The mobile interface can be used as a way to save the experiences youhave when you are mobile. It can be to save a garment you see in a storefor later retrieval at home. The mobile interface can also be capable ofcapturing images of for instance an outfit which you would like torecreate when you come to your larger screen device.

In one example the user uses her mobile phone to access the body model.This can be combined with for example bar code scanning of clothes sheeither would like to see on the model directly on the phone or that shewould like to try on in the virtual fitting room in the store. The phonecan also display styling tips and special offers available to the user.

Section 118: Application Programming Interface (API)

The virtual fitting room has a server side API which can be used forvarious ways of integration. This could for instance be ACTP or JasonAPI as can be used for creating macros and finding information. Thiscould be used for deep integration of the virtual fitting room.

As an alternative the virtual fitting room also can be provided as aJava based service using a java script library. This could be used forthe body model visualization widget for example. However in some casesthere could be reasons why the user should be taken to the virtualfitting room native environment for the body model creation process andthen be taken back to the third party browsing experience with the bodymodel.

Different levels of integration are possible.

1) The first level is to use the virtual fitting room as a visualizationtool for the retailing experience, showing different garments incombination.

2) The next level would be that the retailers would like to pushdifferent trends out through the virtual fitting room.

3) Another aspect of the integration is to integrate personalvisualization elements into email marketing campaigns. This could be toshow the user's body model in the email dressed in a garment that thesender of the email would like to show the user in, for instance formarketing. This step would most likely be through the server side API.

4) The retailers can also enable that the full current outfit the bodymodel is wearing is pulled in to the shopping basket. If the outfitcontains garments of different brands or garments from differentretailers one aspect is that selecting to purchase the whole outfitwould place the relevant garments in different baskets depending onwhere they can be bought.

Section 119: Automatic Detection of Garments and Method and System toTry the Garments on the Body Model Overview

A browser extension that allows a user to “try on” garments using theirbody model where the garments are digitized in the virtual fitting roomsystems, even if the website being viewed has not embedded the virtualfitting room's UI technology.

The Enabling Tech and Data Needed

The user is using a browser that has been customized to provide thesystem. In one example this could be via a browser extension. In analternative example the user can have access to an application with anembedded browser (“Browser Extension”).

The system also consists of an Image Identification Engine thatidentifies that two image resources are of the same scene despiterescaling/changing headers/re-encoding etc.

The system further consists of a process that allows the virtual fittingroom to associate online images of garments that have been digitized bythe virtual fitting room with the virtual fitting room stored model ofthe garment and present that garment on the user's associated bodymodel.

The System

When the user is browsing the Internet, the Browser Extension examinesimages, passing them to the Image Identification Engine to establishwhether the image corresponds to a garment the virtual fitting room hasidentified. If it does identify a garment, the user is given the optionto replace the image with the virtual fitting room's interactive view ofthe garment on the user's body model.

In one example the user can actively select that an image is to bepassed on to the Image Identification engine for garment detection. Theuser can also in one example indicate on a specific image what portionof the image includes the garment which is to be identified. This couldbe done for instance by indicating with a click in the middle of thegarment or for instance by indicating the perimeters of the garment inthe image to assist the Image Identification Engine.

In an alternative example the Browser Extension also examines themetadata for each of the images for an identification of the garment.This identification can be in the form of a standardised system forgarments and/or products. Such a standardised system can be for instancethe International Article Number (EAN), Stock-keeping unit (SKU) orsimilar systems. The identification can also be a proprietary system ofidentification codes unique to each retailer or garment producer. TheImage Identification Engine will have access to different databases foridentification of garments by the embedded identification in the image.

The Image Identification Engine is in one example able to storepreviously identified image locations and/or images and associatedgarments to be presented to the user when they are browsing the web.This speeds up the process of identifying garments and enhances the userexperience.

Section 120: The Similarity System

If a garment in an image of a website the user is browsing is not theexact version of a garment in the virtual fitting room's garment librarythe Image Similarity Engine can semantically characterize the garmentsin the image (e.g. automatically detect that an image is of a redmaxi-dress). The browser extension then offers the user the opportunityto try a similar garment from the virtual fitting room's library on theuser associated body model.

In one example the browser extension automatically selects the mostsimilar garment and presents it on the user's body model.

In an alternative example the browser extension presents the user withalternative garments that match the garment in the identified image andthe user can select which garment is to be presented on the user'sassociated body model.

The Image Similarity Engine uses different parameters to identify andclassify a garment in an image and how similar it is to a storedgarment. It can for instance use what type of garment it is, such as“trousers” or “dress” and also the colour, fabric and if it has sleevesand the length of the sleeves.

In one example the user can also assist the Image Similarity Engine byinputting certain values such as what type of garment it is.

Further Details

The user would typically not need to log in each time they use thebrowser extension. The authentication details would in one example becached and the user's id in the virtual fitting room system and some oftheir body model details could also be cached.

It is further understood that in one example the user can select onwhich of several body models associated with the user's account that anidentified garment is to be presented.

Section 121: Key Features of Data Services

There is provided a method of determining if a shopper is buyinggarments for themselves or for someone else; comprising the steps of:(a) the shopper providing a virtual body model of themselves to acomputer-based system; (b) the shopper purchasing garments at an on-lineor retail store of a garment retailer; (c) the computer-based systemmatching the size of the purchased garments to the virtual body modeland then determining if the purchased garments are for the shopper ornot.

The method may be one in which the computer-based system is linked to adatabase which includes data about the shopper. The method may be one inwhich the database provides loyalty awards, dis-counts or points toshoppers.

There is provided a method of generating search results using a websearch engine, comprising the steps of (a) a user generating a virtualbody model of themselves; (b) the web search engine using that virtualbody model, or data that defines some or all of that virtual body model,in its search process, algorithm or algorithms.

The method may be one so that search results are optimised to berelevant to a user with that particular virtual body model whensearching. The method may be one so that searches for shopping forgarments returns search results that are relevant to a potentialpurchaser with that user's virtual body model. The method may be one inwhich search results can be filtered manually by a user of the searchengine by price and whether or not the user's virtual body model isused. The method may be one in which the search engine is associatedwith a social network and that social network includes the user'svirtual body model. The method may be one in which, if the search engineis associated with such a social network, then the virtual body model isautomatically used in all searches, unless the user has manually set apreference against that. The method may be one in which the virtual bodymodel includes the age, sex, height, weight, and body shape of the user.The method may be one in which the search engine optimises searches froma user for any of the following good or services or events using thevirtual body model of that user: garments, footwear, food, restaurants,holidays, flights, insurance, sports events, medicines, physicaltherapy, books, entertainment.

Section 122: Uses for the Data that is Gathered

What can it be used for: Popular body shapes; Usage statistics;Abandoned baskets.

Also the Facebook data to go through from Facebook. The users couldagree to give this data away.

Gather user information: Sizing from photos; Past purchase history;Likes and dislikes; Demographics.

Intelligent solutions: Size & Fit; Purchase history including:color-ways, styles, sizes; Current trends, inventory; Retailerpromotions

Monetisation: Margin—For goods sold using the Metail solution;Auctions—For direct marketing space in customer communications;Licensing—For using the product; Subscriptions—To data about customersshapes and sizes; And retailer savings from lower returns andphotography costs.

Data in the form of demographic shape, who has what size etc.Understanding what types of shapes buy what types of garments.

There can also be a trending layer on top of the demographic. Not onlywhat shape but also what they actually are buying.

Section 123: Market Research

The data gathered within the virtual fitting room can be used forinstance for market research. Data can be provided for garment specificresearch for example regarding that the garment is tried out by manyusers but is not bought to the same high extent as other garments.

Another aspect of the research data is what garments the users havesaved and for how long they keep the garments saved in their wardrobe orpersonal storage on the platform.

Section 124: Size and Shape Demographic

Understanding which people are buying specific sizes and what thatmatches up against in terms of the demographic of those people. Thisdata can be fed back in to the retailers and can be shared around howtheir fit models actually match up with the body models of theircustomers.

Another aspect is that the virtual fitting room can provide fitting dataand shape data on different markets. The users in different markets havedifferent body shapes. Also users in different markets might havedifferent perceptions on good clothing fit and some like it better looseand some tighter. The data provided from the virtual fitting room canaggregate this data.

Indicators that the body model is actually created to represent the usercan be that the model is used for shopping repetitively or that the bodymodel is being used to a high extent.

Section 125: Online Search

Online search is reliant on good information and the more informationthat is provided the better, and the more relevant the search resultsbecome. Including sizing information in a product search means that thesearch would be more relevant to the user, by providing only garmentsthat match the user's size (or a different size the user is using forsearch).

The virtual fitting room can provide that data to a search engine,either as the sizing information for a specific retailer or brand or forinstance the measurements of the user's body model.

Section 126: Key Features of Garment Morphing Strategies

The following garment morphing strategies may be practised with thesystem.

Section 127: Photographing a Deforming Mannequin to Capture ShapeVariations

We may photograph garments on a deforming mannequin to ensure that wecapture important modes of shape variation. The idea is that theimportant modes of deformation captured can be chosen according togarment properties.

In one example sufficient information is gathered by one photo, or onephoto per view. In another the technology uses a few photos of everyview to collect the information.

Three sizes giving us further points to interpolate likely garmentmorphing behaviour against different sizes and shape is what isrequired. A basic mannequin that changes between a few sizes will giveus the extra bits of information we need, such as the presence ofvertical versus horizontal creases, to then compute and create the restautomatically with our system. We are effectively getting a couple morereal view example points with the mannequin to improve syntheticdeformation performance.

Section 128: Garment Size and Stretch BACKGROUND

Metail solves the problem of online clothing fit by allowing users tocreate 3D photo-real body models of themselves. Metail also seeks toprovide accurate size recommendation to users. This is to help reducenumbers of returns, which are estimated to be between 25% to 40%, formajor online retailers. As part of this, a way to quantify the absolutesize and stretch of garments is required.

Problem

Inconsistency in sizing of garments between manufacturers and designersmeans that generic sizing charts lack accuracy and consistency. We seekto quantify size and stretch of each garment digitized by Metail toprovide an accurate and consistent sizing recommendation to the user.

DESCRIPTION

We apply a force normal to the garments in a vertical plane acrossdefined points in the garment, which correspond to different bodyregions, such as, but not limited to, chest, waist and hips. Furthermorethe force is spread, in each instance, across a region of the garment,by means of a flexible material, to mimic curvature in body shape. Theextension of the garment for each force applied to it is ascertained togive a relationship between the stretch of a garment and a force appliedto it. The force may be applied in a variety of ways including, but notlimited to, manual force, electromagnets and linear actuators.Furthermore we measure the absolute size of garments. These measurementsprovide data to create a metric of different qualities of fit to allowthe user to purchase garments of the correct size.

We provide a consistent, quick and simple way to ascertain data for usein garment size recommendation.

Examples of setups for measuring garment stretch properties are shown inFIG. 1.

Examples

The measuring device may be used at the time of digitizing garments forMetail, to provide a sizing recommendation scheme for that particulargarment. The garment may be placed on the apparatus and the extension ofthe fabric for a series of forces would be measured. The garment mightthen be repositioned a number of times on the apparatus to ascertainmeasurements for other regions of the garment

Other

A mechanical device for measuring absolute garment size and stretch asthe basis for a size recommendation engine. Allows an accurate andconsistent sizing recommendation to be provided to users.

By doing this our solution moves further away from what competitors aredoing today.

Delivering stretch information could be valuable.

On the model, the user can see where it will fit and not fit. This helpsto identify the problematic areas.

We could use this information to provide back into the retail industryto help the design process.

See also Section 68.

Section 129: Key Features of Head Modelling

The system implements the following head modelling features:

See also Section 4.

Section 130: Attaching the Face to the Head Model

When modelling the head, the system uses heads with neck and hair andattaches a face to that head. One aspect of this is that a real hairmodel can be used and that the hair will look more natural.

One part of the user's face which has been defined as containing the keyfeatures is extracted to be used. This part containing the eyes, nose,mouth down to the jaw is then “transplanted” in on to the model head.The ears, neck and hairstyle is kept from the model's head.

The head is then modelled from the 2D image to be viewable from at leastthe eight viewpoints as used in the virtual fitting room.

The shape of the head model's neck is adjusted to fit with the bodymodel's shoulders. This can be done irrespective of if the face has beenreplaced with the features of the user's face.

The head is in one example adjusted in size to match the body model'ssize. A larger body model should have a larger head to have the rightproportions of the overall body.

The number of possible heads for the user to select a hairstyle from isfiltered based on the derived skin tone of the user's uploaded face.

The head is created with support for layers enabling garments thatinterfere with the area occupied by the head. One example is if the bodymodel is wearing a turtle neck, the skin on the neck is behind thegarment while the hair is shown as usual.

When combining artificial elements with representations from pictures,the lighting has to be consistent between the different objects. Theidentification process for lighting includes problems with the whitebalance.

In an example the user holds a white object underneath her head in orderto identify and adjust the white balance.

Section 131: Hair as a Layered Garment

The head with the hair is treated as a layered garment in the system.This way the hair can be displayed together with a garment. The hair isannotated using a similar process to garments. This way long hair cansit on top of a garment: on the shoulder, for instance.

Section 132: Hands as a Layered Garment

Hands and feet can be modelled as a garment. In this way the user canfor instance select to use Kate Moss's hands or someone else's legs orhands. These body parts can be coloured using the same colour value asfor the rest of the body.

In general hands can be quite hard to model digitally and through usinga pair of real hands, the overall appearance of the body model is morenatural.

Machine learning for upload process of face: Machine learning can beused to infer the correct points in a face, eg. eyes, mouth, nose etc.Getting the points closer to the actual position. This speeds up theprocess of uploading your face for the Me Model.

Section 133: Server Infrastructure

The system is implemented with the following computer-based sub-systems:

Section 134: Image/Me Model Rendering on the Back Server

The Image/Me model rendering on the back server and sending images insections to the client makes the client device agnostic.

Section 135: Visualization Subsystem (VS)

Examples of IT infrastructure are provided in FIG. 44 and FIG. 45. Thethroughput of the VS is in an example close to being directlyproportional to the speed per-render.

Section 136: Adding More Nodes

The visualization subsystem is for example homogenous and can workstatelessly in operation. This means we can increase the throughput ofthis subsystem simply by adding more nodes.

Section 137: Caching Strategies

Since the speed-per-render of the subsystem is proportional to itsthroughput, we can improve scalability by employing caching and usingsession affinity in the VS load balancer to optimize access. There are anumber of potential strategies here: —

Section 138: Two-Part Request Affinity

Requests for the RGB and A parts of a render are sent to the sameserver, allowing most of the intermediate artefacts to be cached.

Section 139: Per-Avatar Session Affinity

Directing the same avatar visualizations to the same render node allowsfor high coherence in caching of body model data intermediates. Thisstrategy forces a sub-optimal approach to load balancing if done naively(whereby an individual node could easily become overloaded while therest are idle), so a form of sharding may be used, whereby an avatar isdirected to 2 or 3 nodes from a pool of many.

Section 140: Client-Side Image Caching

The images generated during body model creation are often the sameregardless of the user. By using standard web caching mechanisms (E-Tagheaders, caching pragmas etc), these may be cached in standard webcaches and in users' browsers, leading to lower load on the VS.

Section 141: Garment Data

The database of digital garment models may in an example be small enoughto fit on a single VS node. This will not be the case in some examplesfor which we may take one of the following approaches:

Section 142: Sharding

The garment data is divided (‘sharding’) between the render nodes insuch a way that a load balancer can direct a request to visualize anoutfit to a render node that has the right data.

Section 143: Distributed Database (DB)

The garment database is moved into a separate subsystem that the VSnodes can access over the network (like a traditional Relationaldatabase management system (RDBMS)). It's likely that some form ofcaching strategy would need to be employed in this case to maintainper-render speed.

Section 144: Hardware

A platform example (the Apple MacMini) was selected for having a goodbalance of GPU/CPU/IO performance at a price that represents good value.In another example, Solid-state drive (SSD) devices may be used ratherthan Hard disk drives (HDDs), because these could give better value.Frequent profiling of the software enables one to reason aboutperformance/price trade-offs between hardware alternatives.

Section 145: Web Tier

In an example, the web tier consists of a number of logical components:

AJAX web client (HTML, CSS, JS. A minimal amount of server-sidetemplating): Metail written software.

API engine (Java application running in Jetty): Metail-written software.

DB (MySQL)

Distributed file system

Web server

Load balancer

These components may run on the same hardware, but can all interactthrough network interfaces. A generic route to scalability involvesidentifying those components that are resource-hungry, and moving thosecomponents onto dedicated hardware.

An important property of the API engine is that the Java applicationrunning it is entirely stateless—it does not maintain per-user sessions.This allows us to scale trivially (by adding nodes), whilst being ableto load balance randomly across the tier, without having to worry aboutsession distribution e.g. through multicast.

Section 146: Improved DB Performance

This can be achieved in a number of ways, presented in the order ofimportance bottlenecks may arise:

Query optimization. Profiling may reveal the most performance criticalqueries. By rewriting these queries and adding indices to the database,we can improve speed and throughput.

Moving the database onto dedicated hardware. The DB may run on the samehardware as the rest of the API engine. At some point, a tipping pointis reached where the increased speed of query execution gained byrunning it on separate hardware exceeds the network overhead incurred.At this point the DB can be moved to run on a separate master-slavecluster.

Expand the DB cluster. Database throughput for reads (but not writes)can be improved by increasing the number of slaves in the RDBMSmaster-slave cluster.

Data partitioning—we can separate out garments per retailer, and canshard the end user data between several different databases, or byoptimizing individual nodes for a subset of the data and directingtraffic intelligently.

Move the database onto specialized hardware, with e.g. RAIDed SSDs. Oncethe database is running on its own hardware, a solution to improve queryexecution is to improve the hardware. The problem with this route isthat it has rapidly diminishing cost-performance increase, and there isa hard limit on how much performance can be improved.

Rewrite database layer. By using a distributed database with relaxedconsistency guarantees (e.g. MongoDB, CouchDB), we can improve read andwrite throughput for the database. This might not be necessary forMetail's core application, but it is possible as a strategy for e.g.recommender system and data mining.

Section 147: Optimize and Distribute Performance Critical Code

The performance of the API engine can be improved by the usual profilingand code optimization approaches. In addition to this, the same approachmay be used as used with the VS: identifying performance criticalsections of code, in particular those that are functional (i.e. do notrequire access to the database) and stateless in operation, and creatingsubsystems around them to allow them to be scaled trivially, throughaddition of nodes.

Section 148: Offload SSL Decryption

Our application may make heavy use of SSL to secure communicationsbetween the AJAX web client and the API engine. These requests aredecrypted by the load balancer. At around 200 Mbit/s of traffic, we mayneed to dedicate a pair of machines to the load balancer and install anentropy key service to ensure smooth operation.

Section 149: Spare Capacity/Node Provisioning

An important aspect of scalability is the ability to cope with a suddenhigh-load. This is catered for by keeping spare capacity for variousparts of the system on hot- or warm-standby, such that they can beincluded in the system at short notice, but don't incur cost whilst onstandby. There are a couple of examples of strategies that make thismore effective and practical:

By automating the setup of our servers and optimizing this automaticinstallation, we reduce the time, cost and effort required to expand oursystem, which allows us to scale more quickly, and allows us to carryspare capacity in the form of minimally installed servers.

Disaster resilience will mean that we may keep a mirror of our system ina geographically separate data center. This mirror will be used as sparecapacity by default.

Section 150: Various Miscellaneous Implementation Features

The body model can in one example be duplicated on the screen, so thatthe user can dress the body model and compare two separate outfits sideby side. Another aspect is that a user can select to compare her bodymodel with another user, for instance her boyfriend or a friend to seehow the two outfits match together.

The virtual fitting room may allow the user to search and filter thegarments presented using different filter parameters. For instance theuser can filter the garments based on a specific style, a productcategory or a fashion trend.

The user can select to randomize the hairstyle used on the body model.This allows the user to create a random look on the body model.

In the virtual fitting room the user can wear accessories such as bags,scarfs and hats in different ways. A scarf can for instance be worn tiedaround the neck and alternatively loosely hanging over the shoulders.Similarly the user can select at which height she would like to wear abelt if that is not worn with the trousers. In one example the user canclick and drag the belt into the correct position. Glasses andsunglasses can be worn on the face or up on the head or for instance inthe breast pocket of a shirt.

The virtual fitting room provides personalized fitting information tothe users. This is communicated in a friendly way or in the manner of astylist eg. ‘This is too loose on your bust, try a size smaller’ or‘this is too small for your hips, try a bigger size or an A-line shapeddress’.

The virtual fitting room can provide recommended style advice andcomplementary products that go with what is being worn and also fit withthe style that has been bought by the user in the past. This can forinstance be in the form of a ‘personalise my outfit’ button where theoutfit is automated. This process can be refined through user input onthe recommended garments or outfit. The user can for instance “dislike”or rate a recommended outfit.

In one example the virtual fitting room provides general user stylingtips for the user's shape and size (e.g. You are an hourglass so avoidhigh necks and dropped waistlines; with this dress try wearing a waistbelt to make the most of your figure and highlight your best features).

Users can indicate their best feature and their feature they need tocover up to inform specific recommendation of garments and styling.

In one example the user can select to remove the bra from the bodymodel. The difference that this makes to the garment is then shown.

A virtual fitting room mobile app/widget is an alternative interface inwhich the user can interact with the body model and the virtual fittingroom. For instance the profile can be accessed, looks created andshared, products purchased, comments received from other users, and thefollowing of other users, are possible.

More sliders and input fields may be added to the user interface—forexample an input for inner thigh size. These need not have valuesattached to them, but can be a “fine tune” so users can get the shapethey think is accurate.

Additionally, there can be provided a “fine tune” section where you can(eg. using uncalibrated sliders) change parts of the body after it ismade—such as thigh width, shoulders, how big arms are etc. Examples ofbody parts for which values can be added or fine tuned are Head,Forehead, Jaw, Cheek, Chin, Eye, Ear, Nose, Mouth, Teeth, Neck,Shoulders, Arm, Elbow, Wrist, Hand, Fingers, Thumb, Spine, Chest,Breast, Abdomen, Groin, Hip, Buttocks, Navel, Leg, Thigh, Knee, Calf,Heel, Ankle, Foot and Toes.

All publications mentioned in the above specification are hereinincorporated by reference.

Note

It is to be understood that the above-referenced arrangements are onlyillustrative of the application for the principles of the presentinvention. Numerous modifications and alternative arrangements can bedevised without departing from the spirit and scope of the presentinvention. While the present invention has been shown in the drawingsand fully described above with particularity and detail in connectionwith what is presently deemed to be the most practical and preferredexample(s) of the invention, it will be apparent to those of ordinaryskill in the art that numerous modifications can be made withoutdeparting from the principles and concepts of the invention as set forthherein.

1. Method of generating a 3D virtual body model of a person combinedwith a 3D image of a garment, in which: (a) the 3D virtual body model isgenerated from user data; (b) the 3D garment image is generated using avideo camera to capture multiple views of the garment, and (c) the 3Dgarment image is shown super-imposed over the 3D virtual body model. 2.Method of claim 1, further including sharing the generated 3D virtualbody model of the person combined with the 3D image of the garment. 3.Method of claim 1, in which the garment is chosen by sizerecommendation, or in which the garment size is selected which wouldgive the best fit to the size of the user's body model.
 4. Method ofclaim 1, in which it is provided for the user's body model to try on agarment of a size that is not the best fit.
 5. Method of claim 1, inwhich garment stretch properties are used to generate the 3D garmentimage shown super-imposed over the 3D virtual body model, or in whichfabric specific features are used to generate the 3D garment image shownsuper-imposed over the 3D virtual body model.
 6. Method of claim 1, inwhich a display device displays the 3D garment image shown super-imposedover the 3D virtual body model, and in which the display device providesa user selectable icon, button, function, sliding scale or other controlthat is user selectable to cause the image of the garment to be alteredto correspond to what a garment of a different size would look like onthe virtual body model.
 7. Method of claim 1, in which the garment isimaged with depth information by using a depth sensor that includes asensor which captures video or still data in 3D.
 8. Method of claim 1,in which the garment is imaged with depth information by using a depthsensor that includes an infrared laser projector combined with a sensorwhich captures video or still data in 3D.
 9. Method of claim 1, in whichthe user data includes: (a) a photograph and user-provided measurementdata, or (b) user-provided measurement data.
 10. Method of claim 1, inwhich the image of a garment is a 3D photo-real image of the garment,and in which the 3D photo-real image is one which the user can rotateand/or zoom.
 11. Method of claim 1, in which the virtual body model isaccessible or useable by multiple different applications or multipledifferent web sites, such that images of the garment can, using any ofthese different applications or web sites, be seen as combined onto thevirtual body model to enable visualization of what the garment will looklike when worn.
 12. Method of claim 1, in which the garment image isshown in a virtual fitting room, and the virtual fitting room isaccessible or useable by multiple different applications or multipledifferent web sites, such that images of the garment can, using any ofthese different applications or web sites, be seen as combined onto thevirtual body model to enable visualization of what the garment will looklike when worn.
 13. Method of claim 1, in which it is possible toshow/share a virtual body model to another user so that other user canselect and buy clothes to fit the user, whilst concealing actualmeasurements from that other user.
 14. Method of claim 1, in which thevirtual body model is generated by the person interacting with a website, and in which the virtual body model is accessible or useable by adifferent web site, namely a social networking web site.
 15. Method ofclaim 1, in which the person has selected a garment on a garment retailweb site and that selected garment is visualised on the virtual bodymodel, and in which the garment retail web site includes a button oricon which, when selected, automatically posts an image of the garmentvisualised on the virtual body model to the social networking web sitepage, entry or resource of that person.
 16. Method of claim 1, in whicha social networking web site page, entry or resource of that personenables one or more of the following functions: (i) showing what theuser is wearing or thinking of wearing and/or buying, allowing friendsto comment, express a like; (ii) enabling friends to dress the virtualbody model with their own suggested clothing; (iii) enabling friends toadd their own clothed virtual body model to the user's virtual bodymodel so that the user's page shows both models together in their chosengarments.
 17. Method of claim 1, in which a system can access a virtualbody model of the user and a database of garments or garment images andthe user is able to select from one of several different recommendationchannels, such that, when a specific channel is selected, then thesystem automatically, without user intervention, selects a garment whichthe system determines is suitable given parameters of the virtual bodymodel and then combines an image of that garment onto the virtual bodymodel and automatically without user intervention provides to the useran image of the garment, combined with the virtual body model, to enablethat person to visualise what the garment will look like when worn bythem.
 18. Method of claim 1, in which the virtual body model isgenerated by a user interacting with an application or web site, and inwhich the virtual body model is accessible or useable by any type ofdisplay device that is operable to use an extensible image framework,such that images of a garment can on any such display device be seen ascombined onto the virtual body model, to enable that person to visualisewhat the garment will look like when worn by them.
 19. Method of claim1, in which, when the user is viewing an image of their virtual bodymodel combined with one or more garments on a portable device, the usercan, with a single physical gesture, cause the same image to bedisplayed on a different display device.
 20. Method of claim 19, inwhich the gesture is a flick or throw gesture.
 21. Method of claim 1, inwhich the user's body is imaged using a depth sensor that includes aninfrared laser projector combined with a sensor which captures video orstill data in 3D.
 22. Method of claim 1, in which a user takes, or hastaken for them, an image of their body which is then processed by acomputer system to generate and display a virtual body model using, atleast, the body image and the user is presented with controls thatenable the shape and/or measurements of the virtual body model to bealtered to more accurately match the user's real or perceived shapeand/or measurements, in which the user is presented with an on-screenfield or control that enables the user to provide feedback about theaccuracy of the virtual body model.
 23. Method of claim 1, in which theuser is presented with on-screen controls that allow the user todirectly manipulate the shape of the virtual body model.
 24. Method ofclaim 1, in which finite element analysis is applied to model how thevirtual garment would sit or form itself over the virtual body model ofdefined shape and/or measurements, when values for variables, includingone or more of physical garment elasticity, weight, density, stiffness,material type, thickness, are input.
 25. A system adapted or operable togenerate a 3D virtual body model of a person combined with a 3D image ofa garment, in which: (a) the 3D virtual body model is generated fromuser data; (b) the 3D garment image is generated using a video camera tocapture multiple views of the garment, and (c) the 3D garment image isshown super-imposed over the 3D virtual body model.